Platform comparison mixing libc families

[fauno]

Hi! After #2918 and #2922, I've extended my own gem repository to contain glibc-linked gems also, and found out that in a machine with glibc, where the platform is x86_64-linux, bundler installs x86_64-linux-musl gems as if they were compatible. I checked again and found out this on rubygems/platform.rb:

  ##
  # Does +other+ match this platform?  Two platforms match if they have the
  # same CPU, or either has a CPU of 'universal', they have the same OS, and
  # they have the same version, or either has no version.
  #
  # Additionally, the platform will match if the local CPU is 'arm' and the
  # other CPU starts with "arm" (for generic ARM family support).

  def ===(other)

The "or either has no version" bit was the one that ends up comparing x86_64-linux to x86_64-linux-musl and finding them equal. I patched the version check to @version == other.version and bundler installed the gem from the correct platform, but I'm wondering if it's ok to submit this patch as a PR since the documentation was so specific about it. Could you clarify? Thanks!

This issue is related to:

• Network problems
• Installing a library
• Publishing a library
• The command line gem
• Other

Here are my current environment details:

$ gem env version
3.0.3

I will abide by the code of conduct.

[lloeki]

I report the same issue.

I produced libv8-node builds for the following platforms, published s such:

• x86_64-darwin-19
• x86_64-darwin-20
• x86_64-linux (IOW glibc-based)
• x86_64-linux-musl
• x86_64-solaris-2.11

On the gem production side, just to be able to properly produce the correct x86_64-linux-musl, I had to perform the following hack for ruby versions < 2.7 (I believe this is due to a rubygems change, which turns out to be bundled starting with ruby 2.7):

  def binary_gemspec(platform = Gem::Platform.local, str = RUBY_PLATFORM)
    platform.instance_eval { @version = 'musl' } if str =~ /-musl/ && platform.version.nil?

    gemspec = eval(File.read('libv8-node.gemspec'))
    gemspec.platform = platform
    gemspec
  end

  def binary_gem_name(platform = Gem::Platform.local)
    File.basename(binary_gemspec(platform).cache_file)
  end

On the gem consumption side, gem install libv8-node will install either the -linux one or the -linux-musl one, depending on whatever the order of search results happens to be.

This is a real problem for e.g mini_racer to depend on libv8 and have the distinction made between musl and glibc, and part of why we had to fork it into sq_mini_racer which does not explicitly depends on libv8/libv8-node but has its own search and resolution implementation against the rubygems.org API.

[fauno]

On the gem production side, just to be able to properly produce the correct x86_64-linux-musl, I had to perform the following hack for ruby versions < 2.7 (I believe this is due to a rubygems change, which turns out to be bundled starting with ruby 2.7):

It's due to a previous patch I submitted that was accepted and bundled with ruby 2.7, that made this one come up

[lloeki]

Yup, the hack is only necessary for versions of rubygems before your fix, so thanks for that!

[lloeki]

Regarding the "or either has no version" thing, I think this is e.g for darwin or solaris, so that maybe you can have a gem published as x86_64-darwin or x86_64-solaris and it can be installed regardless of the version, e.g on x86_64-darwin-{17,18,19,20}.

As for linux, while it's possible to build binaries that do run on both glibc and musl, it's fairly involved and requires extreme care (e.g at the very least strict conformance to POSIX and disabling any GNU extension when building it with GCC among other things) which which makes it impractical to assume that linux, nil should match as a wildcard in that case. Also there may be some other libc such as uclibc.

So, pragmatically today, it seems that linux is strictly equal to linux-glibc at large, a fact largely confirmed by Alpine's intentional behaviour of dropping binary gems mentioned by @fauno.

Thus I'm currently attempting a patch, and ultimately a PR, implementing the following:

• preserve the version nil wildcard for non-linux platforms, just in case
• ensure linux === linux-glibc and no other
• ensure linux-foo === linux-foo and no other

[lloeki]

@fauno I opened the PR above, in hopes it would solve this issue, WDYT?

[fauno]

I messed up something with the patch so I'm editing this comment out :)

[fauno]

It works! +1

[lloeki]

@deivid-rodriguez now that #4082 is reverted, can you reopen this issue for better tracking of what we agreed upon? (I'm going to start tackling this this week)

[fauno]

Hi! Please let me know if I can help with something :)

[lloeki]

Sorry folks, it's been a bit of a hellhole for me, so just a ping that I have a lot of things to clear up but I'm not forgetting about this.

[fauno]

Just got notified Alpine 3.15 is going to reenable the musl platform, but not sure which bundler version is going to include: https://gitlab.alpinelinux.org/alpine/aports/-/issues/10808

[lloeki]

It seems like the Ruby 3.x Docker images (which have a recent rubygems + bundler) behave correctly (or at least more correctly than before).

Got some reasonable backport for people stuck with the official Docker images that have rubygems 3.0 + bundler 1.17 and can't update them due to e.g rails 4.x-5.1 bundler < 2.0 constraint.

https://gist.github.com/lloeki/12dcf61324f64a2fa1e8a8b2109c1f00

[lloeki]

at least more correctly than before

Well, there are apparently still some situations where recent rubygems+bundler trip up and decide to fetch linux-musl on linux.

I was starting to wonder if this issue was still relevant, but I guess it definitely is.

[eregon]

Yes, this still happens: #5508
Incorrectly using linux-muslc on glibc systems (the current situation) seems worse than not using -linux gems on Alpine which uses musl (#4434).

Maybe we can do a quick fix that just always prevents -linux-musl to be used on a -linux system? That's always wrong.

[ShepFc3]

I'm experiencing this on the latest ruby alpine image. Are there any workarounds to get this working properly? I've followed all the linked cases and it appears this is still an issue.

[deivid-rodriguez]

This will be fixed by #5852, sorry it took so long!