`bundler install` should not fail when a native platform gem's required_ruby_version is not met, and a suitable `ruby` platform gem is available #4012

flavorjones · 2020-10-09T17:26:20Z

Describe the problem as clearly as you can

Given:

I am running a specific version of Ruby
a Gem is available as both a ruby platform gem and a native (e.g., x86_64-linux) platform gem
- where my specific version of Ruby does not meet the requirements set by the native platform gem via spec.required_ruby_version
- where my specific version of Ruby does meet the requirements set by the ruby platform gem via spec.required_ruby_version

I expect bundler install (and/or gem install?) to install the ruby platform gem.

More specifically, I expect bundler install to iterate through the list of Rubygems Platforms for my system, from more-specific to more-general, until it finds a gem that meets all the requirements. It should not give up and raise an error until the list of platforms has been exhausted.

Instead, I see bundler install select the native gem, and then fail while resolving dependencies with an error message like:

nokogiri-1.11.0.rc3-x86_64-linux requires ruby version < 2.8.dev, >= 2.4, which is incompatible with the current version, ruby 3.0.0p-1

Additional context

Given the work done at rubygems/bundler#7522 to improve platform-specific gem resolution, I expected this to eventually install the matching ruby gem rather than give up after failing to install the non-matching native gem. Perhaps this is my misunderstanding of that issue.

As a maintainer of Nokogiri, about to release native gems (see sparklemotion/nokogiri#2075), I'd like very much for this to work as described. I believe the current behavior will cause confusion for Ruby users whenever a new version of Ruby is made available, and that will create additional support work for native gem maintainers.

I can imagine this unfortunate scenario:

ruby 3.0.0 final is released in December 2020
nokogiri v1.11.0 is simultaneously released with native gems that support ruby ~> 3.0.0
time passes
ruby 3.1 final is released in December 2021 (or, more realistically, ruby 3.1.0preview1 is released in September 2021 and people start trying it out)
developers everywhere try to upgrade and see the above error message
developers open multiple issues on bundler and nokogiri, and post on stackoverflow, and complain on twitter ...
adoption of ruby 3.1.0 is slowed or inhibited

Further confounding the problem is that Ruby users have no way to pin a specific gem to the ruby platform, instead they have to set the bundler config force_ruby_platform and avoid native gems entirely, until each and every gem they use supports the new version of Ruby.

I hope I've explained this scenario adequately -- it's a ticking time bomb once a commonly-used Gem like nokogiri ships native platform gems. Please ask me questions if I'm not being clear, or correct me if my understanding is incorrect.

(I'm tagging @tenderlove, @larskanis, and @casperisfine for visibility because I've discussed this scenario with each of them, and they have context they may wish to share.)

Post steps to reproduce the problem

Brief

on an x86_64-linux machine
install ruby 3.0.0preview1, rubygems 3.2.0.rc1, and bundler 2.2.0.rc1
attempt to install nokogiri =1.11.0.rc3 from rubygems.org using bundler install
see the above error message

Longer

On a linux machine:

Linux juno 5.4.0-48-lowlatency #52-Ubuntu SMP PREEMPT Thu Sep 10 11:51:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Running ruby installed via rvm:

$ ruby -v
ruby 3.0.0preview1 (2020-09-25 master 0096d2b895) [x86_64-linux]

$ gem -v
3.2.0.rc.1

$ bundler -v
Bundler version 2.2.0.rc.1

With these configurations:

$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 3.2.0.rc.1
  - RUBY VERSION: 3.0.0 (2020-09-25 patchlevel -1) [x86_64-linux]
  - INSTALLATION DIRECTORY: /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1
  - USER INSTALLATION DIRECTORY: /home/flavorjones/.gem/ruby/3.0.0
  - RUBY EXECUTABLE: /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/bin/ruby
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1/bin
  - SPEC CACHE DIRECTORY: /home/flavorjones/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-linux
  - GEM PATHS:
     - /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1
     - /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/lib/ruby/gems/3.0.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
     - "gem" => "--no-ri --no-rdoc --no-document"
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1/bin
     - /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1@global/bin
     - /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/bin
     - /home/flavorjones/.rvm/bin
     - <other entries redacted>

$ bundler env
## Environment

...
Bundler       2.2.0.rc.1
  Platforms   ruby, x86_64-linux
Ruby          3.0.0p-1 (2020-09-25 revision 0096d2b895395df5ab8696d3b6d444dc1b7730b6) [x86_64-linux]
  Full Path   /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/bin/ruby
  Config Dir  /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/etc
RubyGems      3.2.0.rc.1
  Gem Home    /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1
  Gem Path    /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1:/home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/lib/ruby/gems/3.0.0
  User Home   /home/flavorjones
  User Path   /home/flavorjones/.gem/ruby/3.0.0
  Bin Dir     /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1/bin
Tools         
  Git         2.25.1
  RVM         1.29.10-next (master)
  rbenv       not installed
  chruby      not installed
...

## Bundler Build Metadata

...
Built At          2020-10-09
Git SHA           unknown
Released Version  false
...

## Bundler settings

...
jobs
  Set for the current user (/home/flavorjones/.bundle/config): 2
gem.test
  Set for the current user (/home/flavorjones/.bundle/config): "rspec"
gem.mit
  Set for the current user (/home/flavorjones/.bundle/config): true
gem.coc
  Set for the current user (/home/flavorjones/.bundle/config): true
force_ruby_platform
  Set for the current user (/home/flavorjones/.bundle/config): false
...

## Gemfile

### Gemfile

...ruby
source "https://rubygems.org"
gem "nokogiri", "=1.11.0.rc3"
...

### Gemfile.lock

...
<No /home/flavorjones/tmp/foobar/Gemfile.lock found>
...

In a project directory with the following Gemfile:

# Gemfile
source "https://rubygems.org"
gem "nokogiri", "=1.11.0.rc3"

Run:

$ bundle install
Fetching gem metadata from https://rubygems.org/.......
Resolving dependencies...
nokogiri-1.11.0.rc3-x86_64-linux requires ruby version < 2.8.dev, >= 2.4, which is incompatible with the current version, ruby 3.0.0p-1

Which command did you run?

bundle install

What were you expecting to happen?

I expected bundler to choose the ruby platform gem rather than the native platform gem.

More specifically, I expect bundler install to iterate through the list of Rubygems Platforms for my system, from more-specific to more-general, until it finds a gem that meets all the requirements. It should not give up and raise an error until the list of platforms has been exhausted.

What actually happened?

Error message:

nokogiri-1.11.0.rc3-x86_64-linux requires ruby version < 2.8.dev, >= 2.4, which is incompatible with the current version, ruby 3.0.0p-1

If not included with the output of your command, run `bundle env` and paste the output below

See above.

The text was updated successfully, but these errors were encountered:

deivid-rodriguez · 2020-10-13T16:30:58Z

Hi @flavorjones!

This is definitely how I expected this to work too after rubygems/bundler#7522 😬.

However, it turns out that this behaviour is only enabled with the specific_platform setting, which will get enabled by default in bundler 3.

Since bundler 2.2.0 is supposed to be the release finally providing a nicer platform support, my proposal would be to enable this setting by default in bundler 2.2.0 instead.

Thoughts?

EDIT: Thanks for the nicely written report by the way, it's really great when people spend time on writing good issues ❤️.

flavorjones · 2020-10-14T14:29:13Z

Somehow I missed the specfic_platform bundler config setting. 🤯

Yes, I personally think we should turn that on in Bundler 2.2 in time for a Ruby 3 release. But my opinion is obviously non-binding, as I'm not a maintainer.

EDIT: I've gone through the installation experience with specific_platform set to true and it worked exactly as I'd hoped.

deivid-rodriguez · 2020-10-15T12:13:25Z

The main inconvenience of this setting is that it will generate lockfile changes by adding the current "specific platform" (x86_64-linux or whatever) to the lockfile. I guess some users could find this annoying, but it seems necessary for proper multiplatform support and other changes we are shipping already go in this direction, like rubygems/bundler#7580, where bundler will automatically add all platforms specified in the Gemfile to the lockfile.

So I'd say let's do this now, but I'd like to hear other thoughts.

If I don't get feedback in a few days I'll go ahead and merge #4015.

deivid-rodriguez · 2020-11-05T13:41:50Z

@flavorjones I didn't forget about this, I'm just fixing some issues with specific_platform before enabling it by default 👍.

Also, regarding

Further confounding the problem is that Ruby users have no way to pin a specific gem to the ruby platform, instead they have to set the bundler config force_ruby_platform and avoid native gems entirely, until each and every gem they use supports the new version of Ruby.

I added #4049 to implement this.

deivid-rodriguez · 2020-11-05T13:43:48Z

With the fixed platform resolution logic #4049 is not as necessary, but I found some cases where it could still be handy.

flavorjones added the Bundler label Oct 9, 2020

flavorjones mentioned this issue Oct 9, 2020

Feedback and final to-do items for precompiled native gems on Linux and OSX sparklemotion/nokogiri#2075

Closed

deivid-rodriguez mentioned this issue Oct 13, 2020

Enable specific_platform by default #4015

Merged

4 tasks

deivid-rodriguez closed this as completed in #4015 Nov 12, 2020

prognostikos mentioned this issue Jan 4, 2021

buildpack bundler version with latest nokogiri (1.11.0 - native gems) fails to deploy heroku/heroku-buildpack-ruby#1105

Closed

flavorjones mentioned this issue Feb 1, 2021

[bug] Can't install nokogiri with Ruby 3.1.0.dev sparklemotion/nokogiri#2185

Closed

flavorjones mentioned this issue Jul 14, 2022

Do not consider platform gems as candidates when required_ruby_version is incompatible #5715

Closed

flavorjones mentioned this issue Oct 20, 2022

Automatically remove "ruby" from lockfile if incomplete #5807

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`bundler install` should not fail when a native platform gem's required_ruby_version is not met, and a suitable `ruby` platform gem is available #4012

`bundler install` should not fail when a native platform gem's required_ruby_version is not met, and a suitable `ruby` platform gem is available #4012

flavorjones commented Oct 9, 2020 •

edited

deivid-rodriguez commented Oct 13, 2020 •

edited

flavorjones commented Oct 14, 2020 •

edited

deivid-rodriguez commented Oct 15, 2020

deivid-rodriguez commented Nov 5, 2020

deivid-rodriguez commented Nov 5, 2020

bundler install should not fail when a native platform gem's required_ruby_version is not met, and a suitable ruby platform gem is available #4012

bundler install should not fail when a native platform gem's required_ruby_version is not met, and a suitable ruby platform gem is available #4012

Comments

flavorjones commented Oct 9, 2020 • edited

Describe the problem as clearly as you can

Additional context

Post steps to reproduce the problem

Brief

Longer

Which command did you run?

What were you expecting to happen?

What actually happened?

If not included with the output of your command, run bundle env and paste the output below

deivid-rodriguez commented Oct 13, 2020 • edited

flavorjones commented Oct 14, 2020 • edited

deivid-rodriguez commented Oct 15, 2020

deivid-rodriguez commented Nov 5, 2020

deivid-rodriguez commented Nov 5, 2020

`bundler install` should not fail when a native platform gem's required_ruby_version is not met, and a suitable `ruby` platform gem is available #4012

`bundler install` should not fail when a native platform gem's required_ruby_version is not met, and a suitable `ruby` platform gem is available #4012

flavorjones commented Oct 9, 2020 •

edited

If not included with the output of your command, run `bundle env` and paste the output below

deivid-rodriguez commented Oct 13, 2020 •

edited

flavorjones commented Oct 14, 2020 •

edited