Platform specific gems not being found by bundler #5743

aericson · 2022-07-19T18:22:24Z

Describe the problem as clearly as you can

Bundler not able to find platform-specific gem after upgrading to 2.3.17 with existing an Gemfile.lock. Getting the same results both on Mac and on linux.

Maybe related to #5715
Maybe introduced by #5495

I'm very new to ruby, please let me know if there is anything else I can do to help debugging this.

Did you try upgrading rubygems & bundler?

Yes, tried 2.3.18 too.

Post steps to reproduce the problem

Add a Gemfile:

# frozen_string_literal: true

source "https://rubygems.org"

gem "sorbet-static-and-runtime"

Add a Gemfile.lock:

GEM
  remote: https://rubygems.org/
  specs:
    sorbet (0.5.10160)
      sorbet-static (= 0.5.10160)
    sorbet-runtime (0.5.10160)
    sorbet-static (0.5.10160-universal-darwin-14)
    sorbet-static (0.5.10160-universal-darwin-15)
    sorbet-static (0.5.10160-universal-darwin-16)
    sorbet-static (0.5.10160-universal-darwin-17)
    sorbet-static (0.5.10160-universal-darwin-18)
    sorbet-static (0.5.10160-universal-darwin-19)
    sorbet-static (0.5.10160-universal-darwin-20)
    sorbet-static (0.5.10160-universal-darwin-21)
    sorbet-static (0.5.10160-x86_64-linux)
    sorbet-static-and-runtime (0.5.10160)
      sorbet (= 0.5.10160)
      sorbet-runtime (= 0.5.10160)

PLATFORMS
  arm64-darwin
  ruby
  x86_64-darwin-20
  x86_64-linux

DEPENDENCIES
  sorbet-static-and-runtime

BUNDLED WITH
   2.3.15

Run bundle install with version 2.3.17+
Run bundle outdated with version 2.3.17+

~ bundler outdated
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Bundler could not find compatible versions for gem "sorbet-static":
  In snapshot (Gemfile.lock):
    sorbet-static (>= 0.5.10160)

  In Gemfile:
    sorbet-static-and-runtime was resolved to 0.5.10160, which depends on
      sorbet (= 0.5.10160) was resolved to 0.5.10160, which depends on
        sorbet-static (= 0.5.10160)

Deleting your Gemfile.lock file and running `bundle install` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "sorbet-static-and-runtime":
  In snapshot (Gemfile.lock):
    sorbet-static-and-runtime (>= 0.5.10160)

  In Gemfile:
    sorbet-static-and-runtime

Deleting your Gemfile.lock file and running `bundle install` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

What were you expecting to happen?

Bundler should tell me there is an update to the gem.

Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...

Gem                        Current    Latest     Requested    Groups
sorbet                     0.5.10160  0.5.10175
sorbet-runtime             0.5.10160  0.5.10175
sorbet-static              0.5.10160  0.5.10175
sorbet-static-and-runtime  0.5.10160  0.5.10175  = 0.5.10160  default

If not included with the output of your command, run `bundle env` and paste the output below

Environment

Bundler       2.3.17
  Platforms   ruby, arm64-darwin-21
Ruby          3.1.2p20 (2022-04-12 revision 4491bb740a9506d76391ac44bb2fe6e483fec952) [arm64-darwin-21]
  Full Path   /Users/aericson/.asdf/installs/ruby/3.1.2/bin/ruby
  Config Dir  /Users/aericson/.asdf/installs/ruby/3.1.2/etc
RubyGems      3.3.7
  Gem Home    /Users/aericson/projects/bundlerissue/vendor/bundle/ruby/3.1.0
  Gem Path    /Users/aericson/projects/bundlerissue/vendor/bundle/ruby/3.1.0
  User Home   /Users/aericson
  User Path   /Users/aericson/.local/share/gem/ruby/3.1.0
  Bin Dir     /Users/aericson/projects/bundlerissue/vendor/bundle/ruby/3.1.0/bin
Tools
  Git         2.37.0
  RVM         not installed
  rbenv       not installed
  chruby      not installed

Bundler Build Metadata

Built At          2022-06-29
Git SHA           539b20c172
Released Version  true

Bundler settings

path
  Set for your local app (/Users/aericson/projects/bundlerissue/.bundle/config): "vendor/bundle"
  Set for the current user (/Users/aericson/.bundle/config): "vendor/bundle"
[REDACTED]
  Set for the current user (/Users/aericson/.bundle/config): "token:[REDACTED]"

The text was updated successfully, but these errors were encountered:

deivid-rodriguez · 2022-07-19T18:29:02Z

Interesting. I just tried this with Bundler 2.3.18 and everything worked as expected, and I'm using the exact same platform and ruby version you are using. Can you upgrade RubyGems too, since that's the only difference I can see between our environments?

aericson · 2022-07-19T19:49:12Z

@deivid-rodriguez thanks for trying, that's really odd. I just tried with the updated gems and the same issue:

Bundler       2.3.18
  Platforms   ruby, arm64-darwin-21
Ruby          3.1.2p20 (2022-04-12 revision 4491bb740a9506d76391ac44bb2fe6e483fec952) [arm64-darwin-21]
  Full Path   /Users/aericson/.asdf/installs/ruby/3.1.2/bin/ruby
  Config Dir  /Users/aericson/.asdf/installs/ruby/3.1.2/etc
RubyGems      3.3.18
  Gem Home    /Users/aericson/projects/bundlerissue/vendor/bundle/ruby/3.1.0
  Gem Path    /Users/aericson/projects/bundlerissue/vendor/bundle/ruby/3.1.0
  User Home   /Users/aericson
  User Path   /Users/aericson/.local/share/gem/ruby/3.1.0
  Bin Dir     /Users/aericson/projects/bundlerissue/vendor/bundle/ruby/3.1.0/bin
Tools
  Git         2.37.0
  RVM         not installed
  rbenv       not installed
  chruby      not installed

I'm only able to test this on a Mac M1 right now.

Something interesting I just found out, if I manually edit the Gemfile.lock and remove the ruby from the platform list it started to work again. Seems that somehow the ruby one is taking priority and failing.

*** a	Tue Jul 19 21:42:44 2022
--- b	Tue Jul 19 21:39:07 2022
***************
*** 20,23 ****
--- 20,24 ----
  PLATFORMS
    arm64-darwin
-   ruby
    x86_64-darwin-20
    x86_64-linux

deivid-rodriguez · 2022-07-19T20:04:32Z

I reproduced it now. Not sure why it didn't reproduce before. I had some development version of Rubygems + Bundler installed, my optimistic theory is that I had some WIP work to fix #5715 and that's also fixing this one 😅.

Anwyays, I agree this might be caused by #5495.

I'll work on this.

deivid-rodriguez · 2022-07-21T06:28:03Z

Hei, so I had a look at this yesterday and... I'm pretty sure this is indeed #5495 and what's worse, it's somehow expected. The rationale is that the Gemfile.lock is invalid for the "ruby" platform, because "ruby" essentially means that no platform specific gems should be used, and that's not possible for sorbet-static (in other words, the lockfile can't be used with force_ruby_platform).

However, this strict meaning of the "ruby" platform should only apply to freshly generated lockifles, and in #5495 I only intended to raise this error to avoid generating new "incorrect" lockfiles, but did not intend to break using old lockfiles with this shape. So, your case should be restored to work again and at most warn.

That said, it seems tricky to fix so my advice to workaround for now is to do what you already did, remove the "ruby" platform from the lockfile.

aericson · 2022-07-21T08:28:09Z

Thanks for looking into this so quick! That is indeed unfortunate, we have quite a few places where we have this issue. I understand that this was an unintended consequence of the other fix.

I'm not sure if it's a good idea, but is it worth reverting #5495 and reintroducing it on a non-patch version or if a way is found to introduce this as a warning? The reason I bring this up is that I see more people reaching this issue internally and seems like people can end up spending a lot of time debugging this as the cause might not be straight forward. I'm not familiar with #5495 so maybe it's not worth the trade-off reverting it, just putting the thought out there.

Anyway, thanks for looking into this! I'll remove the ruby line.

deivid-rodriguez · 2022-07-21T09:57:24Z

No problem!

#5495 was not a very important fix, but it introduced some simplifications that I think unblock fixing other issues, like #4488. So I'd like to not revert it.

I'll try to figure out a way to keep existing lockfiles working and either print a warning in this case, or directly remove the "ruby" platform from them.

ashkulz · 2022-07-22T15:19:53Z

@deivid-rodriguez Dependabot seems to have upgraded the bundler version they use internally and all our repositories which use sorbet now fail with this error 😞

aericson · 2022-07-22T15:22:49Z

@deivid-rodriguez Dependabot seems to have upgraded the bundler version they use internally and all our repositories which use sorbet now fail with this error 😞

@ashkulz yea, that was pretty much how we found out about this too. Removing ruby from the platforms list should fix it though.

deivid-rodriguez · 2022-07-22T15:23:18Z

Sorry about that. As I said before I'll try to figure this out. At least you have a workaround, that's something.

ashkulz · 2022-07-22T15:32:48Z

@aericson yeah, I found that workaround too. Not sure if it's safe to use it, though ... will have to QA it properly on Monday before merging.

technicalpickles · 2022-07-28T17:04:32Z

I'm actually running into this issue, even though we do not have ruby in the platforms. It was in our monolith, but I was able to extract just enough to reproduce: https://github.com/technicalpickles/bundler-platform-test

We are not getting the platform-specific version of grpc for linux specifically, and that's been causing some performance hits during CI and other places we bundle install from scratch. I tried re-adding the platform (ie bundle lock --add-platform x86_64-linux), but that doesn't seem to change the lock file.

If I start from scratch, ie rm Gemfile.lock, bundle install, and add the platforms, it works as expected.

deivid-rodriguez · 2022-07-28T19:13:18Z

Can you confirm it's also a regression from #5495? If it is, what was the previous behavior, exactly?

kbarrette · 2022-08-01T21:49:53Z

I have just a single platform ruby in my Gemfile.lock, and neither leaving it in nor removing it allows me to update from bundler 2.3.16 to .19 - bundler complains about not being able to resolve sorbet-static and can't continue. Dependabot is also of course broken, because they use .18

deivid-rodriguez · 2022-08-02T07:08:38Z

I'll work on this soon. To workaround your case I think you can regenerate your lockfile, so it uses your current platform instead of "ruby".

kbarrette · 2022-08-02T13:10:11Z

I'll work on this soon. To workaround your case I think you can regenerate your lockfile, so it uses your current platform instead of "ruby".

Removing Gemfile.lock and running bundle install just gives me the same error message

deivid-rodriguez · 2022-08-02T13:12:15Z

Oh, this might be a different issue then. Can you show your Gemfile?

kbarrette · 2022-08-02T13:22:31Z

Oh, this might be a different issue then. Can you show your Gemfile?

❯ ls
Gemfile

❯ ruby -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-darwin20]

❯ bundle -v
Bundler version 2.3.19

❯ cat Gemfile
ruby '3.1.2'

source 'https://rubygems.org'

gem 'sorbet-static-and-runtime'

❯ bundle
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies.............
Bundler could not find compatible versions for gem "sorbet-static":
  In Gemfile:
    sorbet-static-and-runtime was resolved to 0.5.9519, which depends on
      sorbet (= 0.5.9519) was resolved to 0.5.9519, which depends on
        sorbet-static (= 0.5.9519)

Could not find gem 'sorbet-static (= 0.5.9519)' with platform 'ruby', which is required by gem 'sorbet (= 0.5.9519)', in rubygems repository https://rubygems.org/ or installed locally.

The source contains the following gems matching 'sorbet-static (= 0.5.9519)':
  * sorbet-static-0.5.9519-java
  * sorbet-static-0.5.9519-universal-darwin-14
  * sorbet-static-0.5.9519-universal-darwin-15
  * sorbet-static-0.5.9519-universal-darwin-16
  * sorbet-static-0.5.9519-universal-darwin-17
  * sorbet-static-0.5.9519-universal-darwin-18
  * sorbet-static-0.5.9519-universal-darwin-19
  * sorbet-static-0.5.9519-universal-darwin-20
  * sorbet-static-0.5.9519-universal-darwin-21
  * sorbet-static-0.5.9519-x86_64-linux

❯

deivid-rodriguez · 2022-08-02T13:30:09Z

Yeah, it's a different issue, but I guess also caused by #5495. Which gem do you expect to get installed, and which gem did you get installed before? It should be sorbet-static-0.5.9519-universal-darwin-20 I guess?

deivid-rodriguez · 2022-08-02T13:32:58Z

Actually, can you show the result of bundle config? I suspect you might have force_ruby_platform set. In that case, that would be expected and precisely the bug fixed by #5495. And you can fix it by unsetting force_ruby_platform.

kbarrette · 2022-08-02T13:42:46Z

Actually, can you show the result of bundle config? I suspect you might have force_ruby_platform set. In that case, that would be expected and precisely the bug fixed by #5495. And you can fix it by unsetting force_ruby_platform.

Yep, I do have force_ruby_platform set. No idea when or why I set it, but I'll experiment with turning it off. Thanks!

deivid-rodriguez · 2022-08-03T18:37:56Z

#5807 should fix this!

aericson added the Bundler label Jul 19, 2022

aericson changed the title ~~Platform specific gems not being found after upgrading to 2.3.17~~ Platform specific gems not being found by bundler Jul 19, 2022

deivid-rodriguez added the status: feedback required label Jul 19, 2022

deivid-rodriguez added type: bug report and removed status: feedback required labels Jul 19, 2022

deivid-rodriguez mentioned this issue Jul 26, 2022

Dependabot has stopped opening PRs - breaks with sorbet dependencies dependabot/dependabot-core#5420

Closed

jez mentioned this issue Aug 1, 2022

JSON::ParserError on running srb init sorbet/sorbet#6134

Closed

deivid-rodriguez mentioned this issue Aug 2, 2022

Automatically remove "ruby" from lockfile if incomplete #5807

Merged

4 tasks

deivid-rodriguez closed this as completed in #5807 Aug 5, 2022

technicalpickles mentioned this issue Feb 6, 2023

I have the same problem technicalpickles/bundler-platform-test#1

Closed

fabianrbz mentioned this issue Mar 7, 2024

chore(deps-dev): bump rack from 3.0.8 to 3.0.9.1 kumahq/kuma-website#1674

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Platform specific gems not being found by bundler #5743

Platform specific gems not being found by bundler #5743

aericson commented Jul 19, 2022

deivid-rodriguez commented Jul 19, 2022

aericson commented Jul 19, 2022

deivid-rodriguez commented Jul 19, 2022

deivid-rodriguez commented Jul 21, 2022

aericson commented Jul 21, 2022

deivid-rodriguez commented Jul 21, 2022 •

edited

ashkulz commented Jul 22, 2022

aericson commented Jul 22, 2022

deivid-rodriguez commented Jul 22, 2022

ashkulz commented Jul 22, 2022

technicalpickles commented Jul 28, 2022

deivid-rodriguez commented Jul 28, 2022

kbarrette commented Aug 1, 2022 •

edited

deivid-rodriguez commented Aug 2, 2022

kbarrette commented Aug 2, 2022

deivid-rodriguez commented Aug 2, 2022

kbarrette commented Aug 2, 2022 •

edited

deivid-rodriguez commented Aug 2, 2022

deivid-rodriguez commented Aug 2, 2022

kbarrette commented Aug 2, 2022

deivid-rodriguez commented Aug 3, 2022

Platform specific gems not being found by bundler #5743

Platform specific gems not being found by bundler #5743

Comments

aericson commented Jul 19, 2022

Describe the problem as clearly as you can

Did you try upgrading rubygems & bundler?

Post steps to reproduce the problem

What were you expecting to happen?

If not included with the output of your command, run bundle env and paste the output below

Environment

Bundler Build Metadata

Bundler settings

deivid-rodriguez commented Jul 19, 2022

aericson commented Jul 19, 2022

deivid-rodriguez commented Jul 19, 2022

deivid-rodriguez commented Jul 21, 2022

aericson commented Jul 21, 2022

deivid-rodriguez commented Jul 21, 2022 • edited

ashkulz commented Jul 22, 2022

aericson commented Jul 22, 2022

deivid-rodriguez commented Jul 22, 2022

ashkulz commented Jul 22, 2022

technicalpickles commented Jul 28, 2022

deivid-rodriguez commented Jul 28, 2022

kbarrette commented Aug 1, 2022 • edited

deivid-rodriguez commented Aug 2, 2022

kbarrette commented Aug 2, 2022

deivid-rodriguez commented Aug 2, 2022

kbarrette commented Aug 2, 2022 • edited

deivid-rodriguez commented Aug 2, 2022

deivid-rodriguez commented Aug 2, 2022

kbarrette commented Aug 2, 2022

deivid-rodriguez commented Aug 3, 2022

If not included with the output of your command, run `bundle env` and paste the output below

deivid-rodriguez commented Jul 21, 2022 •

edited

kbarrette commented Aug 1, 2022 •

edited

kbarrette commented Aug 2, 2022 •

edited