Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TruffleRuby's support for Childprocess gem #1525

Closed
deepj opened this issue Dec 9, 2018 · 11 comments
Closed

TruffleRuby's support for Childprocess gem #1525

deepj opened this issue Dec 9, 2018 · 11 comments
Assignees
Labels
ffi third-party-issue An issue which should be fixed upstream

Comments

@deepj
Copy link

deepj commented Dec 9, 2018

I'm not sure, if this is the right place to put in here. childprocess is used by selenium-webdriver (see https://rubygems.org/gems/childprocess/reverse_dependencies)

selenium-webdriver gem is used by many testing libraries such as capybara. childprocess is not under active maintenance. In fact, a new maintainer is needed (enkessler/childprocess#131).

The issue is, it uses fork under TruffleRuby (which is not supported).

Backtrace:

NotImplementedError: fork is not available
    <internal:core> core/kernel.rb:764:in `fork’
    truffleruby-head/gems/childprocess-4.1.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process’
    truffleruby-head/gems/childprocess-4.1.0/lib/childprocess/abstract_process.rb:81:in `start’

Workaround: export CHILDPROCESS_POSIX_SPAWN=1, see #1525 (comment)

@eregon
Copy link
Member

eregon commented Dec 10, 2018

childprocess looks very similar to Process.spawn.
I had a quick look at the backends:

Probably the best solution here is adding a backend using Process.spawn, which is the official Ruby core API for spawning subprocesses.

@deepj deepj closed this as completed Jun 8, 2020
@eregon eregon reopened this Jun 19, 2020
@colorfulsing
Copy link

hi guys, first of all, I want to thank you all for the amazing work that you guys have done with truffleruby.

Is there any update on this feature?

I have a heavy threaded app that uses selenium web driver within 50 threads that would really benefit by truffleruby. I have been looking forward to implement it on this app for a the last 2 years :)

@eregon
Copy link
Member

eregon commented Dec 4, 2020

@colorfulsing Did you see any issue from childprocess, can you post it here or in a gist?

Now we have FFI implemented, so I think the posix_spawn backend should work as good as on MRI.
The mentioned race above also exists on MRI, so I guess it's not any worse on TruffleRuby.

@eregon
Copy link
Member

eregon commented Dec 4, 2020

Actually the posix_spawn backend is not the default:
https://github.com/enkessler/childprocess/blob/c6260739e86c1cabd9baebd76f2058eea2b1ae16/lib/childprocess.rb#L73

$ cruby -rchildprocess -e 'p process = ChildProcess.build("echo", "foo")'                                                  
#<ChildProcess::Unix::ForkExecProcess ...

If we activate it explicitly:

$ CHILDPROCESS_POSIX_SPAWN=1 jt ruby -rchildprocess -e 'p process = ChildProcess.build("echo", "foo"); process.io.inherit!; process.start; process.wait'
<internal:core> core/type.rb:105:in `coerce_to_failed': Coercion error: #<ChildProcess::Unix::PosixSpawnProcess::Argv:0xb48 @ptrs=[#<Truffle::FFI::MemoryPointer address=0x7f95ccd0fe30>, #<Truffle::FFI::MemoryPointer address=0x7f95ccd1a6c0>, #<Truffle::FFI::Pointer address=0x0>]>.to_ptr => Truffle::FFI::Pointer failed (TypeError)
	from <internal:core> core/type.rb:93:in `execute_coerce_to'
	from <internal:core> core/type.rb:85:in `coerce_to'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:203:in `get_pointer_value'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:155:in `convert_ruby_to_native'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:81:in `block in call'
	from <internal:core> core/enumerable.rb:354:in `each_with_index'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:80:in `call'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:120:in `posix_spawnp'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:51:in `block (2 levels) in launch_process'
	from <internal:core> core/dir.rb:301:in `chdir'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:43:in `block in launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `synchronize'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/abstract_process.rb:81:in `start'
	from -e:1:in `<main>'
<internal:core> core/truffle/ffi/pointer_access.rb:684:in `put_array_of_pointer': wrong number of arguments (given 2, expected 1) (ArgumentError)
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:103:in `to_ptr'
	from <internal:core> core/type.rb:91:in `execute_coerce_to'
	from <internal:core> core/type.rb:85:in `coerce_to'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:203:in `get_pointer_value'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:155:in `convert_ruby_to_native'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:81:in `block in call'
	from <internal:core> core/enumerable.rb:354:in `each_with_index'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:80:in `call'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:120:in `posix_spawnp'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:51:in `block (2 levels) in launch_process'
	from <internal:core> core/dir.rb:301:in `chdir'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:43:in `block in launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `synchronize'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/abstract_process.rb:81:in `start'
	from -e:1:in `<main>'

Which sounds like a FFI issue, we'll try to look at that.

The better solution would be to use Process.spawn though, so I'll file an issue on the gem repo for that (enkessler/childprocess#172).

@colorfulsing
Copy link

hi @eregon , thank you very much for the quick response.

I'm currently testing using standalone truffleruby 20.3.0, childprocess 3.0.0 and selenium 3.142.7 within a docker container using image ruby:2.6.6-slim-stretch as base and I have also unset both GEM_HOME and GEM_PATH env vars so thruffleruby works correctly using it's own gem path.

I have also installed these packages on the docker image:

curl
build-essential
libpq-dev
default-libmysqlclient-dev
apt-transport-https
gettext-base
sqlite3
make
clang
llvm
libc++-dev
libc++abi-dev
libssl-dev
libz-dev
libxslt-dev
libxml2-dev
yarn
nodejs
chromium
chromium-driver
xvfb

And also installed pkg-config gem on truffleruby, here is the log:

/root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:176:in `stop_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:123:in `stop'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:115:in `stop'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/platform.rb:150:in `block in exit_hook'
        from <internal:core> core/exception.rb:126:in `backtrace'
        from <internal:core> core/exception.rb:126:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'
/root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:183:in `stop_server'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:118:in `stop'
        from <internal:core> core/truffle/exception_operations.rb:113:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:763:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `new'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:113:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:119:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:763:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `new'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:113:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:119:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'

And this is the log when using truffleruby-head from yesterday:

/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:38: warning: already initialized constant FFI::Platform::OS
/root/truffleruby-head/lib/truffle/ffi/platform.rb:38: warning: previous definition of OS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:59: warning: already initialized constant FFI::Platform::OSVERSION
/root/truffleruby-head/lib/truffle/ffi/platform.rb:59: warning: previous definition of OSVERSION was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:61: warning: already initialized constant FFI::Platform::CPU
/root/truffleruby-head/lib/truffle/ffi/platform.rb:61: warning: previous definition of CPU was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:63: warning: already initialized constant FFI::Platform::ARCH
/root/truffleruby-head/lib/truffle/ffi/platform.rb:63: warning: previous definition of ARCH was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:91: warning: already initialized constant FFI::Platform::IS_GNU
/root/truffleruby-head/lib/truffle/ffi/platform.rb:91: warning: previous definition of IS_GNU was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:92: warning: already initialized constant FFI::Platform::IS_LINUX
/root/truffleruby-head/lib/truffle/ffi/platform.rb:92: warning: previous definition of IS_LINUX was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:93: warning: already initialized constant FFI::Platform::IS_MAC
/root/truffleruby-head/lib/truffle/ffi/platform.rb:93: warning: previous definition of IS_MAC was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:94: warning: already initialized constant FFI::Platform::IS_FREEBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:94: warning: previous definition of IS_FREEBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:95: warning: already initialized constant FFI::Platform::IS_NETBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:95: warning: previous definition of IS_NETBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:96: warning: already initialized constant FFI::Platform::IS_OPENBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:96: warning: previous definition of IS_OPENBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:97: warning: already initialized constant FFI::Platform::IS_DRAGONFLYBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:97: warning: previous definition of IS_DRAGONFLYBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:98: warning: already initialized constant FFI::Platform::IS_SOLARIS
/root/truffleruby-head/lib/truffle/ffi/platform.rb:98: warning: previous definition of IS_SOLARIS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:99: warning: already initialized constant FFI::Platform::IS_WINDOWS
/root/truffleruby-head/lib/truffle/ffi/platform.rb:99: warning: previous definition of IS_WINDOWS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:100: warning: already initialized constant FFI::Platform::IS_BSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:100: warning: previous definition of IS_BSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:105: warning: already initialized constant FFI::Platform::NAME
/root/truffleruby-head/lib/truffle/ffi/platform.rb:105: warning: previous definition of NAME was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:106: warning: already initialized constant FFI::Platform::CONF_DIR
/root/truffleruby-head/lib/truffle/ffi/platform.rb:106: warning: previous definition of CONF_DIR was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:110: warning: already initialized constant FFI::Platform::LIBPREFIX
/root/truffleruby-head/lib/truffle/ffi/platform.rb:110: warning: previous definition of LIBPREFIX was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:119: warning: already initialized constant FFI::Platform::LIBSUFFIX
/root/truffleruby-head/lib/truffle/ffi/platform.rb:119: warning: previous definition of LIBSUFFIX was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:131: warning: already initialized constant FFI::Platform::LIBC
/root/truffleruby-head/lib/truffle/ffi/platform.rb:131: warning: previous definition of LIBC was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:32: warning: already initialized constant FFI::USE_THIS_PROCESS_AS_LIBRARY
/root/truffleruby-head/lib/truffle/ffi/library.rb:32: warning: previous definition of USE_THIS_PROCESS_AS_LIBRARY was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:32: warning: already initialized constant FFI::CURRENT_PROCESS
/root/truffleruby-head/lib/truffle/ffi/library.rb:32: warning: previous definition of CURRENT_PROCESS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:79: warning: already initialized constant FFI::Library::CURRENT_PROCESS
/root/truffleruby-head/lib/truffle/ffi/library.rb:79: warning: previous definition of CURRENT_PROCESS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:80: warning: already initialized constant FFI::Library::LIBC
/root/truffleruby-head/lib/truffle/ffi/library.rb:80: warning: previous definition of LIBC was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:182: warning: already initialized constant FFI::Library::FlagsMap
/root/truffleruby-head/lib/truffle/ffi/library.rb:182: warning: previous definition of FlagsMap was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/struct_layout_builder.rb:100: warning: already initialized constant FFI::StructLayoutBuilder::NUMBER_TYPES
/root/truffleruby-head/lib/truffle/ffi/struct_layout_builder.rb:100: warning: previous definition of NUMBER_TYPES was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/version.rb:2: warning: already initialized constant FFI::VERSION
/root/truffleruby-head/lib/truffle/ffi/version.rb:2: warning: previous definition of VERSION was here
200
/root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:176:in `stop_process'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:123:in `stop'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:115:in `stop'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/platform.rb:150:in `block in exit_hook'
        from <internal:core> core/exception.rb:126:in `backtrace'
        from <internal:core> core/exception.rb:126:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'
/root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:183:in `stop_server'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:118:in `stop'
        from <internal:core> core/truffle/exception_operations.rb:138:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:758:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:138:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:144:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:758:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:138:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:144:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'

Let me know if I can help you with anything else, and thanks again for the help! :D

@colorfulsing
Copy link

It seems that it requires fork to be implemented on core/kernel.rb, not sure how it works to be honest.

@eregon
Copy link
Member

eregon commented Dec 4, 2020

Yes, the fork+exec backend of childprocess has no chance to work on TruffleRuby, JRuby and on Windows, fork() is simply not implementable there (well, maybe on TruffleRuby+Native but it would be total overkill for the purpose of childprocess).
That's actually something we need to fix in the childprocess gem, it should check Process.respond_to?(:fork) before attempting to use the fork+exec backend, or just always pick the posix-spawn backend on TruffleRuby.

You can set CHILDPROCESS_POSIX_SPAWN=1 to force the posix-spawn backend, but then you'll most likely see the same issue as I posted above (#1525 (comment)). That's probably relatively easy to solve on the TruffleRuby side.

@colorfulsing
Copy link

I see, that's fine, I'm ok to add new env vars on our servers to avoid fork+exec, so I guess I will need to wait a little longer until the other issue is fixed before using truffleruby.

I'm really eager tho, that means that there is not much left before I'm able to implement it on our app :D

Thanks for the info!

@eregon eregon self-assigned this Dec 8, 2020
graalvmbot pushed a commit that referenced this issue Dec 8, 2020
graalvmbot pushed a commit that referenced this issue Dec 8, 2020
@eregon
Copy link
Member

eregon commented Dec 10, 2020

The fix in the TruffleRuby FFI backend was fairly trivial: ff8619d
So now

CHILDPROCESS_POSIX_SPAWN=1 truffleruby -rchildprocess -e 'p process = ChildProcess.build("echo", "foo"); process.io.inherit!; process.start; process.wait'

works.

What's left is auto-detecting this in childprocess and/or enkessler/childprocess#172 so it'd work without any special setting (CHILDPROCESS_POSIX_SPAWN=1 is needed until then).

@eregon
Copy link
Member

eregon commented Nov 14, 2022

selenium-webdriver no longer uses childprocess (in upcoming releases) and the test suite passes on truffleruby 🎉 SeleniumHQ/selenium#11251 (comment)

eregon added a commit to eregon/childprocess that referenced this issue Nov 14, 2022
… fork

* Such as on TruffleRuby, JRuby, etc.
* ForkExecProcess does not work at all on these platforms anyway.
* PosixSpawnProcess is not thread-safe with regard to the current working
  directory (oracle/truffleruby#1525 (comment))
  but it's still better than failing with no fork.
* enkessler#172 is the proper solution longer-term.
sds pushed a commit to enkessler/childprocess that referenced this issue Dec 13, 2023
… fork

* Such as on TruffleRuby, JRuby, etc.
* ForkExecProcess does not work at all on these platforms anyway.
* PosixSpawnProcess is not thread-safe with regard to the current working
  directory (oracle/truffleruby#1525 (comment))
  but it's still better than failing with no fork.
* #172 is the proper solution longer-term.
sds pushed a commit to enkessler/childprocess that referenced this issue Dec 13, 2023
… fork

* Such as on TruffleRuby, JRuby, etc.
* ForkExecProcess does not work at all on these platforms anyway.
* PosixSpawnProcess is not thread-safe with regard to the current working
  directory (oracle/truffleruby#1525 (comment))
  but it's still better than failing with no fork.
* #172 is the proper solution longer-term.
@eregon eregon added the third-party-issue An issue which should be fixed upstream label Jan 7, 2024
@eregon
Copy link
Member

eregon commented Jan 7, 2024

This is now properly fixed in childprocess 5 🎉
enkessler/childprocess#175
https://github.com/enkessler/childprocess/releases/tag/v5.0.0

@eregon eregon closed this as completed Jan 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ffi third-party-issue An issue which should be fixed upstream
Projects
None yet
Development

No branches or pull requests

3 participants