Error "update_depot_tools_toggle.py" building libv8 ruby gem #310

stefanahman · 2020-12-10T11:30:38Z

Hello,

I'm trying to build the libv8 ruby gem in our CI. All of a sudden, I'm getting this error:

________ running 'vpython third_party/depot_tools/update_depot_tools_toggle.py
--disable' in
'/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/vendor'
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/vendor/depot_tools/.cipd_bin/.cipd/pkgs/0/fI6WggdkRyN1r91MnTeApc2_gyTtXfYpueHZVLcaF8gC/vpython:
could not resolve options: failed to resolve Python script: stat
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/vendor/third_party/depot_tools/update_depot_tools_toggle.py:
no such file or directory
Error: Command 'vpython third_party/depot_tools/update_depot_tools_toggle.py
--disable' returned non-zero exit status 1 in
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/vendor
Running: gclient root
Running: gclient config --spec 'solutions = [
  {
    "name": "v8",
    "url": "https://chromium.googlesource.com/v8/v8.git",
    "deps_file": "DEPS",
    "managed": False,
    "custom_deps": {},
  },
]
'
Running: gclient sync --with_branch_heads
Subprocess failed with return code 2.
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/ext/libv8/builder.rb:83:in
`block in setup_build_deps!': unable to fetch v8 source (RuntimeError)
from
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/ext/libv8/builder.rb:81:in
`chdir'
from
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/ext/libv8/builder.rb:81:in
`setup_build_deps!'
from
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/ext/libv8/builder.rb:40:in
`build_libv8!'
from
/home/semaphore/dos/vendor/bundle/ruby/2.5.0/gems/libv8-8.4.255.0/ext/libv8/location.rb:24:in
`install!'
	from extconf.rb:7:in `<main>'

extconf failed, exit code 1

The text was updated successfully, but these errors were encountered:

jdelStrother · 2020-12-10T11:56:10Z

Weirdly, ours has also suddenly started failing in our Github Actions CI.

  /home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/ext/libv8
  /opt/hostedtoolcache/Ruby/2.6.6/x64/bin/ruby -I
  /opt/hostedtoolcache/Ruby/2.6.6/x64/lib/ruby/2.6.0 -r
  ./siteconf20201210-8200-18jih01.rb extconf.rb
  creating Makefile
  WARNING: Your metrics.cfg file was invalid or nonexistent. A new one will be
  created.
  
  ________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout
  --progress https://chromium.googlesource.com/v8/v8.git

....

  Traceback (most recent call last):
  File
  "/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/depot_tools/metrics.py",
  line 266, in print_notice_and_exit
      yield
  File
  "/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/depot_tools/gclient.py",
  line 3155, in <module>
      sys.exit(main(sys.argv[1:]))
  File
  "/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/depot_tools/gclient.py",
  line 3141, in main
      return dispatcher.execute(OptionParser(), argv)
  File
  "/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/depot_tools/subcommand.py",
  line 252, in execute
      return command(parser, args[1:])
  File
  "/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/depot_tools/gclient.py",
  line 2699, in CMDsync
      ret = client.RunOnDeps('update', args)
  File
  "/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/depot_tools/gclient.py",
  line 1755, in RunOnDeps
      gn_args_dep.WriteGNArgsFile()
  File
  "/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/depot_tools/gclient.py",
  line 1022, in WriteGNArgsFile
      with open(os.path.join(self.root.root_dir, self._gn_args_file), 'w') as f:
  IOError: [Errno 2] No such file or directory:
  '/home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/vendor/build/config/gclient_args.gni'
  Running: gclient root
  Running: gclient config --spec 'solutions = [
    {
      "url": "https://chromium.googlesource.com/v8/v8.git",
      "managed": False,
      "name": "v8",
      "deps_file": "DEPS",
      "custom_deps": {},
    },
  ]
  '
  Running: gclient sync --with_branch_heads
  Subprocess failed with return code 1.
  /home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/ext/libv8/builder.rb:83:in
  `block in setup_build_deps!': unable to fetch v8 source (RuntimeError)
  from
  /home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/ext/libv8/builder.rb:81:in
  `chdir'
  from
  /home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/ext/libv8/builder.rb:81:in
  `setup_build_deps!'
  from
  /home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/ext/libv8/builder.rb:40:in
  `build_libv8!'
  from
  /home/runner/work/audioboom-web/audioboom-web/vendor/bundle/ruby/2.6.0/gems/libv8-7.3.492.27.1/ext/libv8/location.rb:24:in
  `install!'
  	from extconf.rb:7:in `<main>'

I'm not sure why it's suddenly started trying to build libv8 from source rather than using the binary, still investigating... 😕

jdelStrother · 2020-12-10T12:07:33Z

Ugh, think it's because bundler 2.2 has just come out, and it's fetching the build-from-source gem rather than the prepackaged binary.

stefanahman · 2020-12-10T12:40:22Z

Ye, I'm locking bundler to version 2.1.4 for now. Thanks @jdelStrother!

deivid-rodriguez · 2020-12-11T08:58:26Z

Hi!

I'm really sorry the bundler release caused issues for you 🙏.

And I don't know why this is happening, because if anything, the new version was expected to choose prepackaged binaries more smartly, but it sounds like it's doing the exact opposite here.

What steps can I run to see the issue for myself and investigate it?

filipkis · 2020-12-11T09:40:48Z

For me somehow installing gem install bundler -v 2.1.4 was not enough (this is on Github Actions workflow so there shouldn't be any other bundler there). The bundler install command still said Using bundler 2.2.0. However running it with bundler _2.1.4_ install finally worked.

deivid-rodriguez · 2020-12-11T09:42:58Z

I think that's because ruby/setup-ruby installs the latest bundler automatically for you. You can opt out of that by passing bundler: none to the action, and then manually installing 2.1.4.

jdelStrother · 2020-12-11T10:32:08Z

What steps can I run to see the issue for myself and investigate it?

I'm finding it surprisingly hard to come up with a small reproduction. Anyone else had better luck? @stefanahman you're not also using ruby/setup-ruby in Github Actions, right?

JunichiIto · 2020-12-11T10:36:41Z

You can opt out of that by passing bundler: none to the action, and then manually installing 2.1.4.

bundler: "2.1.4" works instead of passing bundler: none to the action, and then manually installing 2.1.4.

e.g.

- name: Set up Ruby
  uses: ruby/setup-ruby@v1
  with:
    # NOTE: Bundler 2.2.0 fails to install libv8
    # https://github.com/rubyjs/libv8/issues/310
    bundler: "2.1.4"
    bundler-cache: true

UPDATE
ruby/setup-ruby#127 allows setup-ruby to install the same Bundler version as Gemfile.lock with bundler: "Gemfile.lock" option.

- name: Set up Ruby
  uses: ruby/setup-ruby@v1
  with:
    bundler: "Gemfile.lock"
    bundler-cache: true

deivid-rodriguez · 2020-12-11T10:42:30Z

Oh neat, I didn't know that feature!

deivid-rodriguez · 2020-12-11T10:46:16Z

Maybe it was added recently, because from the current docs I wouldn't expect that to be supported.

travis ci builds are failing with libv8 compilation errors. It might be because bundler 2.2 was released yesterday and we've been installing bundler on travis with "--pre". Something seems to be up with bundler 2.2 being "smart" about fetching pre-built gems. Maybe. rubyjs/libv8#310 (comment)

stefanahman · 2020-12-12T12:19:26Z

What steps can I run to see the issue for myself and investigate it?

I'm finding it surprisingly hard to come up with a small reproduction. Anyone else had better luck? @stefanahman you're not also using ruby/setup-ruby in Github Actions, right?

No sorry, I'm using a different CI without Github Actions.

sandrods · 2020-12-13T21:45:09Z

Same problem here, installing libv8 8.4.255.0 (x86_64-linux) inside a ruby2.6.6-slim buster image, bundler 2.2

paulgoetze · 2020-12-14T11:10:49Z

It looks like you'd need to update the Gemfile.lock to include all platforms you want to support: https://bundler.io/blog/2020/12/09/bundler-v2-2.html (Multiplatform support).

I had the same issues as desribed above (dev environment was on Mac, GitHub actions CI on Linux), and after bundling with bundler 2.2.0 on the specific plattforms I noticed the PLATFORMS were updated in te Gemfile.lock, which also fixed the CI for me:

...
   libv8 (7.3.492.27.1)
+  libv8 (7.3.492.27.1-x86_64-darwin-19)
+  libv8 (7.3.492.27.1-x86_64-linux)
…

PLATFORMS
  ruby
+  x86_64-darwin-19
+  x86_64-linux
…

BUNDLED WITH
-  2.1.4
+  2.2.0

deivid-rodriguez · 2020-12-14T11:17:58Z

Yes, I think I get the issue now.

If you have a lockfile generated with an old version of bundler (where platform specific versions were not recorded in the lockfile, but "resolved" at installation time), then bundler 2.2 will see a generic RUBY platform in the lockfile and try to install that (and thus compile native extensions, etc).

Wherever that lockfile lives, updating it to what bundler 2.2 generates should fix the issue.

However, this was an unintentional incompatibiliity so I'm restoring it with rubygems/rubygems#4127.

stefanahman · 2020-12-14T14:03:55Z

Rebuilding Gemfile.lock after installning bundler v2.2 (with bundle update X) and bundle lock --add-platform x86_64-linux did it for me

2called-chaos · 2021-03-23T17:32:21Z

It looks like you'd need to update the Gemfile.lock to include all platforms you want to support:

Should we include generic ruby still? When I create the lock file from scratch it only lists x86_64-darwin-19 but if someone with a different platform tries to bundle it results in something like this:

Unable to find a spec satisfying libv8 (~> 8.4.255) in the set. Perhaps the lockfile is corrupted? Found libv8 (8.4.255.0-x86_64-darwin-19)
that did not match the current platform.

When I do add ruby it will not say it's corrupted but libv8 will attempt to install from source (and fail after downloading 1GB each time). So for an open source application that means maintaining an ever growing list of platforms? Or instruct something like this as a setup step?

bundle lock --add-platform $(\ruby -e 'puts Gem::Platform.local')

I guess this is more a problem of libv8 scripts at the end of the day but I'm running out of ideas over at this PR

Bundler 2.2 build from source instead of using the prepackaged binary. See: rubyjs/libv8#310

This fixes libv8 installation under GitHub action containers. Bundler 2.2 build from source instead of using the prepackaged binary. See: rubyjs/libv8#310

- rubygems/rubygems#4513 - rubyjs/libv8#310 (comment) - rubygems/rubygems#4497 - https://heroku.support/978905

* Bundler 2.2.16 - rubygems/rubygems#4513 - rubyjs/libv8#310 (comment) - rubygems/rubygems#4497 - https://heroku.support/978905 * Apply suggestions from code review Co-authored-by: Ed Morley <501702+edmorley@users.noreply.github.com> Co-authored-by: Ed Morley <501702+edmorley@users.noreply.github.com>

rubyjs/libv8#310

* install just the dev package of libxslt * removes trailing spaces * migrates some of our travis configs * database config * retire travis task to avoid double started solr/fedora * remove 2.5 add updated shoulda-matchers * stick to bundler 2.1.4 rubyjs/libv8#310 * only build against 2.6 for now * add optional 2.7 job * reconfigure coveralls * adds simplecov * require simplecov

lloeki · 2023-12-04T12:18:28Z

Closing as this is way outdated, plus depot_tools is a PITA, which libv8-node sidesteps.

hsbt mentioned this issue Dec 10, 2020

Bundler does not find platform specific gem rubygems/rubygems#4123

Closed

jdelStrother mentioned this issue Dec 11, 2020

Locking to a specific bundler version ruby/setup-ruby#123

Closed

sethaj mentioned this issue Dec 11, 2020

HELIO-3668 fix failing travis ci builds mlibrary/heliotrope#2889

Merged

ggrossetie mentioned this issue Dec 12, 2020

JavaScript engines are using UTF-16 internally but String are UTF-8 encoded by default opal/opal#2117

Merged

lucidkit mentioned this issue Jan 9, 2021

Error Installing Forem on Linux "installing libv8 (8.4.255.0), and Bundler cannot continue" forem/forem#12189

Closed

citizen428 mentioned this issue Jan 14, 2021

Update Gemfile.lock for Bundler 2.2 forem/forem#12272

Merged

3 tasks

This was referenced Mar 18, 2021

Libv8 error when deploying errbit to heroku errbit/errbit#1494

Open

[WiP] Replace therubyracer with mini_racer (updates v8) errbit/errbit#1496

Merged

gbp added a commit to mysociety/foi-for-councils that referenced this issue Mar 29, 2021

Use bundler 2.1 on CI

92a87ae

Bundler 2.2 build from source instead of using the prepackaged binary. See: rubyjs/libv8#310

gbp added a commit to mysociety/alaveteli that referenced this issue Mar 31, 2021

Use bundler 2.1 on CI

eaf4290

This fixes libv8 installation under GitHub action containers. Bundler 2.2 build from source instead of using the prepackaged binary. See: rubyjs/libv8#310

gbp mentioned this issue Mar 31, 2021

Use bundler 2.1 on CI mysociety/alaveteli#6177

Merged

themilkman mentioned this issue Apr 7, 2021

Bundler 2.2.15 doesn't use prepackaged libv8 version; tries to build locally rubygems/rubygems#4513

Closed

schneems added a commit to heroku/heroku-buildpack-ruby that referenced this issue Apr 14, 2021

Bundler 2.2.16

b2845c6

- rubygems/rubygems#4513 - rubyjs/libv8#310 (comment) - rubygems/rubygems#4497 - https://heroku.support/978905

schneems mentioned this issue Apr 14, 2021

Bundler 2.2.16 heroku/heroku-buildpack-ruby#1150

Merged

mkorcy added a commit to mkorcy/epigaea-1 that referenced this issue Sep 7, 2021

stick to bundler 2.1.4

e3d1db2

rubyjs/libv8#310

tbolender mentioned this issue Nov 10, 2021

Support for Apple Silicon (arm64-darwin21) #319

Closed

lloeki closed this as completed Dec 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error "update_depot_tools_toggle.py" building libv8 ruby gem #310

Error "update_depot_tools_toggle.py" building libv8 ruby gem #310

stefanahman commented Dec 10, 2020

jdelStrother commented Dec 10, 2020 •

edited

jdelStrother commented Dec 10, 2020

stefanahman commented Dec 10, 2020

deivid-rodriguez commented Dec 11, 2020 •

edited

filipkis commented Dec 11, 2020

deivid-rodriguez commented Dec 11, 2020

jdelStrother commented Dec 11, 2020

JunichiIto commented Dec 11, 2020 •

edited

deivid-rodriguez commented Dec 11, 2020

deivid-rodriguez commented Dec 11, 2020

stefanahman commented Dec 12, 2020

sandrods commented Dec 13, 2020 •

edited

paulgoetze commented Dec 14, 2020 •

edited

deivid-rodriguez commented Dec 14, 2020 •

edited

stefanahman commented Dec 14, 2020

2called-chaos commented Mar 23, 2021

lloeki commented Dec 4, 2023

Error "update_depot_tools_toggle.py" building libv8 ruby gem #310

Error "update_depot_tools_toggle.py" building libv8 ruby gem #310

Comments

stefanahman commented Dec 10, 2020

jdelStrother commented Dec 10, 2020 • edited

jdelStrother commented Dec 10, 2020

stefanahman commented Dec 10, 2020

deivid-rodriguez commented Dec 11, 2020 • edited

filipkis commented Dec 11, 2020

deivid-rodriguez commented Dec 11, 2020

jdelStrother commented Dec 11, 2020

JunichiIto commented Dec 11, 2020 • edited

deivid-rodriguez commented Dec 11, 2020

deivid-rodriguez commented Dec 11, 2020

stefanahman commented Dec 12, 2020

sandrods commented Dec 13, 2020 • edited

paulgoetze commented Dec 14, 2020 • edited

deivid-rodriguez commented Dec 14, 2020 • edited

stefanahman commented Dec 14, 2020

2called-chaos commented Mar 23, 2021

lloeki commented Dec 4, 2023

jdelStrother commented Dec 10, 2020 •

edited

deivid-rodriguez commented Dec 11, 2020 •

edited

JunichiIto commented Dec 11, 2020 •

edited

sandrods commented Dec 13, 2020 •

edited

paulgoetze commented Dec 14, 2020 •

edited

deivid-rodriguez commented Dec 14, 2020 •

edited