mini_racer 0.12.0 Segmentation fault on x86_64-linux with libv8-node 21.7.2.0 #300
Comments
|
Hey @D-system. A couple of question hopefully helping to narrow the issue down
|
|
I don't think jemalloc is enabled: $ ruby -r rbconfig -e "puts RbConfig::CONFIG['MAINLIBS']"
-lz -lrt -lrt -ldl -lcrypt -lm -lpthreadEven by using the $ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 ruby -r rbconfig -e "puts RbConfig::CONFIG['MAINLIBS']"
-lz -lrt -lrt -ldl -lcrypt -lm -lpthread
# Checking the directory
$ ls -l /usr/lib/x86_64-linux-gnu/libjemalloc*
-rw-r--r-- 1 root root 1119814 Jan 24 2022 /usr/lib/x86_64-linux-gnu/libjemalloc.a
-rw-r--r-- 1 root root 1119970 Jan 24 2022 /usr/lib/x86_64-linux-gnu/libjemalloc_pic.a
lrwxrwxrwx 1 root root 16 Jan 24 2022 /usr/lib/x86_64-linux-gnu/libjemalloc.so -> libjemalloc.so.2
-rw-r--r-- 1 root root 744440 Jan 24 2022 /usr/lib/x86_64-linux-gnu/libjemalloc.so.2Let me see if I can make a new app and get the same error. I didn't realized there was a more debug output:
```
-- Ruby level backtrace information ---------------------------------------- -- Machine register context ------------------------------------------------ -- C level backtrace information ------------------------------------------- |
|
I made a brand new Rails 6.1.7.7 app I got the same error with those changes:
Note: it does precompile the files then seg fault.
```
$ bundle ; ./bin/rails assets:clobber assets:precompile
Bundle complete! 18 Gemfile dependencies, 84 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
I, [2024-04-23T18:59:24.147478 #10018] INFO -- : Removed /home/circleci/new_rails_app/public/assets
yarn install v1.22.19
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.42s.
I, [2024-04-23T18:59:25.739888 #10018] INFO -- : Writing /home/circleci/new_rails_app/public/assets/manifest-b4bf6e57a53c2bdb55b8998cc94cd00883793c1c37c5e5aea3ef6749b4f6d92b.js
I, [2024-04-23T18:59:25.740896 #10018] INFO -- : Writing /home/circleci/new_rails_app/public/assets/manifest-b4bf6e57a53c2bdb55b8998cc94cd00883793c1c37c5e5aea3ef6749b4f6d92b.js.gz
I, [2024-04-23T18:59:25.741133 #10018] INFO -- : Writing /home/circleci/new_rails_app/public/assets/application-04024382391bb910584145d8113cf35ef376b55d125bb4516cebeb14ce788597.css
I, [2024-04-23T18:59:25.742186 #10018] INFO -- : Writing /home/circleci/new_rails_app/public/assets/application-04024382391bb910584145d8113cf35ef376b55d125bb4516cebeb14ce788597.css.gz
./bin/rails: [BUG] Segmentation fault at 0x00007fb3f7901008
ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [x86_64-linux]
-- Control frame information ----------------------------------------------- -- Machine register context ------------------------------------------------ -- C level backtrace information ------------------------------------------- -- Other runtime information -----------------------------------------------
[...] |
|
I also tried the same project with mini_racer 0.9. It works. |
|
It seems sprocket is used by default if no other assets pipeline exists. The Gemfile without the comments source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
ruby '3.2.3'
gem 'rails', '~> 6.1.7', '>= 6.1.7.7'
gem 'sqlite3', '~> 1.4'
gem 'puma', '~> 5.0'
gem 'sass-rails', '>= 6'
gem 'turbolinks', '~> 5'
gem 'jbuilder', '~> 2.7'
gem "bootstrap", "~> 4.3" # <- the problem
gem "mini_racer", "~> 0.9.0" # <- added
gem 'bootsnap', '>= 1.4.4', require: false
group :development, :test do
gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
end
group :development do
gem 'web-console', '>= 4.1.0'
gem 'rack-mini-profiler', '~> 2.0'
gem 'listen', '~> 3.3'
gem 'spring'
end
group :test do
gem 'capybara', '>= 3.26'
gem 'selenium-webdriver', '>= 4.0.0.rc1'
gem 'webdrivers'
end
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby] |
|
@lloeki this looks urgent , should we roll out a release that puts us back at an earlier version while we work this out ? |
|
@D-system does this repro in Linux as well |
|
@SamSaffron it's all tested inside the CircleCI image linked above. So, it's an Ubuntu based linux on x86_64. |
So the idea would be to have 0.12.1 be 0.9.0? That would be odd. |
|
I'll try a repro. |
|
The official |
|
I'm not able to reproduce yet, can't spend much time on it right now though. The product I'm working on in my day job is using docker images based on the 🤔 |
|
I tried with I'm not able to reproduce the issue in a local docker either. |
In any case it appears that
I couldn't reproduce it either inside that |
|
@D-system any chance you can create a container that repros this consistently. Then we can |
|
I just wanted to add another case of this issue. I tried to update yesterday from 0.8 to 0.12 and started getting segfaults. I don't use docker or any version managers, just the system's 3.0.2p107 (the default ruby) on Ubuntu 22.04. This is on rails 7.0.8.1 with libv8 21.7.2.0. I've reverted to 0.8 and libv8 18.16.0.0 for now. I'm not using jemalloc (I have no idea what it is, and the mentioned libraries are not on my system). If you need any info or need anything tested let me know, but I don't want to figure out how to build or use docker images (have no interest in that). |
|
I also saw it yesterday when upgrading from 0.9 to 0.12 both on Fedora 40 with ruby 3.3.0 and also on Ubuntu 22.04 with ruby 3.0.2p107. |
|
Thanks for the additional reports. Is it Sprockets-related as well? |
|
It was using rails with sprockets yes, specifically using https://rubygems.org/gems/rtlcss in the sprockets chain which uses mini_racer to run https://www.npmjs.com/package/rtlcss on CSS files. |
|
I am using sprockets, but I had not checked whether it's involved or not. I first got these errors when rendering a js response to an ajax request, and also saw them when shutting down the server (running puma). |
|
@SamSaffron I was not able to reproduce on my local host or local docker yesterday. I will give a new try tomorrow. |
|
We were getting this on 0.9.0 -> 0.12.0 (production only) on Heroku on the '20' image with Ruby 3.3.0 and jemalloc and node I did see this in a prod console, though may not be related
and looking at prod crash logs
we're just using this to run embedded javascript
|
Are you using puma in clustered mode? forking is known to cause some troubles with mini_racer/v8, see https://github.com/rubyjs/mini_racer#fork-safety. |
|
I am indeed using puma in clustered mode, but it was working fine on 0.8.0, and starting puma in single mode gives me the same segfault in the same place. I have the suggested fork safety line in an initializer, and removing this makes no difference in any scenario. |
|
I'm still not able to reproduce it in a local Docker even when I use the Docker image of the CI. I continue to investigate on the CI directly. SSH and vi for the win! Anyway, I track down the issue. On a brand new Rails 6.1 app:
I don't know what else I could search. |
|
@D-system I wish I had more to add here, but so much depends on a standalone repro. In theory most people use mini_racer in a very simple way, if you can get a crash then you can monkey patch eval to save what it is evaling into a file and then we can use that to cause the crash. Once we have one we will very likely be able to sort this out quickly There is a chance here as well that libraries were using mini racer in a fork unsafe way and somehow got away with it in earlier versions due to luck. |
|
The same happened to me after upgrading to the latest libv8-node (21.7.2.0) and mini_racer (0.12.0) during running specs
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] |
|
After some lengthy monkey patching session, I'm 99% sure it's due to this file: https://github.com/ai/autoprefixer-rails/blob/master/vendor/autoprefixer.js The file is 6.29MB which isn't the simplest to debug. |
|
Oh wow, does an eval of this file cause the crash?
…On Wed, 1 May 2024 at 6:28 PM, Thomas ***@***.***> wrote:
After some lengthy monkey patching session, I'm 99% sure it's due to this
file:
https://github.com/ai/autoprefixer-rails/blob/master/vendor/autoprefixer.js
I removed the source map at the last line, it does not change anything.
The file is 6.29MB which isn't the simplest to debug.
—
Reply to this email directly, view it on GitHub
<#300 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAABIXPXZDI2KX724Z3GTFTZACRTFAVCNFSM6AAAAABGUEE3QSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBYGE2DQNZUHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I copy/paste that giant file into a browser console and got an error: 
Which is used at line 9900 (other references are inside a strings) |
|
From Chrome 92 (and Firefox too), https://web.dev/articles/cross-origin-isolation-guide https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer |
|
Thanks for diving this deep into it @D-system, IIUC this still not reproduced locally, only in CircleCI? |
I think what Sam meant is mimicking https://github.com/ai/autoprefixer-rails/blob/cf0af2807108e43f4858574b5d22dfaec0630d80/lib/autoprefixer-rails/processor.rb#L133C9-L133C23 i.e does the following cause a crash? If that's not enough, maybe it can be reproduced by mimicking https://github.com/ai/autoprefixer-rails/blob/cf0af2807108e43f4858574b5d22dfaec0630d80/lib/autoprefixer-rails/processor.rb#L35 with a specific input file? Notes:
|
|
I don't know if it's the same bug, but recently I added a new plugin to the Gemfile of a template repository, which caused the Docker image to be rebuilt with the latest versions of the libraries. The build failed in GitHub actions, with the message: |
|
@george-gca entirely unrelated. see rubyjs/libv8-node#36 (comment) |
|
Thank you for dealing.
|
|
I don't have time to investigate further at the moment. I'll be AFK from tomorrow until after the RubyKaigi (2024-05-15 to 17). |
Hello the team
Thank you for the update to libv8-node 21.7.2.0 yesterday.
I wanted to report an error.
Setup:
RAILS_ENV=production SECRET_KEY_BASE=1 bin/rails assets:clobber assets:precompileIt does work on Darwin Kernel Version 23.2.0 arm64 (Mac on arm) that compile from source.
The error is from eval -> eval_unsafe, so I suspect it is an issue with the
libv8-nodemini_racer/lib/mini_racer.rb
Line 228 in 5b2a25c
The text was updated successfully, but these errors were encountered: