intermittent SEG FAULT when running CI tests for listen gem

[ColinDKelley]

We recently added truffleruby to the CI test suite for the listen gem. A recent build failed there with a SEG FAULT.

The backtrace indicates truffleruby/21.0.0-dev-f8b272f9

@gogainda suggested filing a ticket over here.

E, [2020-11-05T23:07:54.053000 #2835] ERROR -- : Exception rescued in listen-run_thread:
Errno::EFAULT: Bad address
<internal:core> core/errno.rb:48:in `handle'
<internal:core> core/posix.rb:356:in `read_string_at_least_one_byte'
<internal:core> core/io.rb:2011:in `readpartial'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:319:in `readpartial'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:290:in `read_events'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:263:in `process'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:236:in `block in run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:232:in `synchronize'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:232:in `run'
/home/runner/work/listen/listen/lib/listen/adapter/linux.rb:41:in `_run'
/home/runner/work/listen/listen/lib/listen/adapter/base.rb:79:in `block in start'
/home/runner/work/listen/listen/lib/listen/thread.rb:24:in `rescue_and_log'
/home/runner/work/listen/listen/lib/listen/thread.rb:17:in `block in new'
--- Thread.new ---
/home/runner/work/listen/listen/lib/listen/adapter/base.rb:75:in `start'
/home/runner/.rubies/truffleruby-head/lib/mri/forwardable.rb:230:in `start'
/home/runner/work/listen/listen/lib/listen/listener.rb:69:in `block in <class:Listener>'
/home/runner/work/listen/listen/lib/listen/fsm.rb:121:in `instance_eval'
/home/runner/work/listen/listen/lib/listen/fsm.rb:121:in `call'
/home/runner/work/listen/listen/lib/listen/fsm.rb:102:in `transition_with_callbacks!'
/home/runner/work/listen/listen/lib/listen/fsm.rb:69:in `transition'
/home/runner/work/listen/listen/lib/listen/listener.rb:90:in `start'
/home/runner/work/listen/listen/spec/acceptance/listen_spec.rb:12:in `block (2 levels) in <top (required)>'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:455:in `instance_exec'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:455:in `instance_exec'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:365:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:529:in `block in run_owned_hooks_for'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:528:in `each'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:528:in `run_owned_hooks_for'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:615:in `block in run_example_hooks_for'
<internal:core> core/array.rb:873:in `reverse_each'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:614:in `run_example_hooks_for'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:484:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:502:in `run_before_example'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:261:in `block in run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:508:in `block in with_around_and_singleton_context_hooks'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:465:in `block in with_around_example_hooks'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:486:in `block in run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:626:in `block in run_around_example_hooks_for'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:350:in `call'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:350:in `call'
/home/runner/work/listen/listen/spec/acceptance/listen_spec.rb:19:in `block (4 levels) in <top (required)>'
/home/runner/work/listen/listen/spec/support/fixtures_helper.rb:22:in `fixtures'
/home/runner/work/listen/listen/spec/acceptance/listen_spec.rb:19:in `block (3 levels) in <top (required)>'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:455:in `instance_exec'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:455:in `instance_exec'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:390:in `execute_with'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:350:in `call'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:350:in `call'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:627:in `run_around_example_hooks_for'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/hooks.rb:486:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:465:in `with_around_example_hooks'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:508:in `with_around_and_singleton_context_hooks'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example.rb:259:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:644:in `block in run_examples'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:640:in `map'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:640:in `run_examples'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:606:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `block in run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `map'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `block in run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `map'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `block in run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `map'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/example_group.rb:607:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:121:in `map'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/configuration.rb:2067:in `with_suite_hooks'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:116:in `block in run_specs'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/reporter.rb:74:in `report'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:115:in `run_specs'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:89:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:71:in `run'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/lib/rspec/core/runner.rb:45:in `invoke'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/gems/rspec-core-3.10.0/exe/rspec:4:in `<top (required)>'
<internal:core> core/kernel.rb:401:in `load'
<internal:core> core/kernel.rb:401:in `load'
/home/runner/work/listen/listen/vendor/bundle/truffleruby/21.0.0-dev-f8b272f9/bin/rspec:23:in `<top (required)>'
<internal:core> core/kernel.rb:401:in `load'
<internal:core> core/kernel.rb:401:in `load'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:63:in `kernel_load'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:28:in `run'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:476:in `exec'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/exe/bundle:46:in `block in <top (required)>'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/home/runner/.rubies/truffleruby-head/lib/gems/gems/bundler-2.1.4/exe/bundle:34:in `<top (required)>'
<internal:core> core/kernel.rb:401:in `load'
<internal:core> core/kernel.rb:401:in `load'
/home/runner/.rubies/truffleruby-head/bin/bundle:23:in `<main>'

More details here:
guard/listen#507

[eregon]

Thanks for the report.
Interesting, we've seen a very similar issue on one of the benchmarks, but only in the Native CE runtime configuration, with MultiTier enabled (GR-27140):

$ mxbuild/truffleruby-native/jre/languages/ruby/bin/truffleruby -S bin/rails routes
<internal:core> core/errno.rb:48:in `handle': Bad address (Errno::EFAULT)
	from <internal:core> core/posix.rb:356:in `read_string_at_least_one_byte'
	from <internal:core> core/io.rb:148:in `fill'
	from <internal:core> core/io.rb:173:in `block in fill_from'
	from <internal:core> core/io.rb:168:in `synchronized'
	from <internal:core> core/io.rb:168:in `fill_from'
	from <internal:core> core/io.rb:1816:in `read'
...

man 2 read on Linux says:

       EFAULT buf is outside your accessible address space.

So maybe the buffer address we give is somehow invalid.
We'll investigate.

I think it's not a good idea to use both --engine.MultiTier --engine.Mode=latency though.
So I would recommend only using --engine.Mode=latency (or no options if it doesn't make a big difference).
--engine.MultiTier is still being stabilized, especially on Native.

[gogainda]

I was playing around with different options to make it run within 6 min. Based on the latest run it looks like truffleruby's become much faster and those parameters are not needed anymore
https://github.com/guard/listen/runs/1367971525

[gogainda]

Corresponding PR guard/listen#511

[eregon]

@gogainda Thanks!

I believe that will solve the issue, so I'll close this.
Feel free to reopen if it happens again.