rubygems-update 3.2.13 breaks sorbet installation on alpine

[garybethea]

My current problem is that using rubygems-update 3.2.13, running bundle install on Alpine fails with the error, Could not find sorbet-static-0.5.6317-universal-darwin-14 in any of the sources even though it is present in Gemfile.lock.

This issue is related to:

• Network problems
• [ X] Installing a library
• Publishing a library
• The command line gem
• Other

Here are my current environment details:

$ gem env version
Platform: ruby:2.6.5-alpine
rubygems-update: 3.2.13
bundler: 2.1.4

Output

#!/bin/bash -eo pipefail
aws_region="HIDDEN"
image_prefix="HIDDEN"

.circleci/build_container \
  --image-prefix "${image_prefix}" \
  --git-sha "${CIRCLE_SHA1}" \

Login Succeeded
Sending build context to Docker daemon  8.201MB
Step 1/22 : FROM ruby:2.6.5-alpine as base
 ---> 59d4ec426ed6
Step 2/22 : LABEL maintainer "HIDDEN@HIDDEN.com"
 ---> Using cache
 ---> ae9448a3f518
Step 3/22 : ARG BUNDLE_GITHUB__COM
 ---> Using cache
 ---> b3cb033adb4e
Step 4/22 : ARG BUNDLER_VERSION=2.1.4
 ---> Using cache
 ---> 061715bc2d57
Step 5/22 : ARG APP_PATH=/app
 ---> Using cache
 ---> b7f859aa0ffa
Step 6/22 : ENV BUNDLER_VERSION ${BUNDLER_VERSION}
 ---> Using cache
 ---> c268d0e3dfa6
Step 7/22 : RUN mkdir -p ${APP_PATH}
 ---> Using cache
 ---> 8e723f4ca6d9
Step 8/22 : WORKDIR ${APP_PATH}
 ---> Using cache
 ---> ca1144a4e2d2
Step 9/22 : RUN apk upgrade && apk add --no-cache   build-base   postgresql-dev   git   less   nodejs-current   tzdata   bash   snappy-dev   libexecinfo   automake   autoconf   libtool
 ---> Using cache
 ---> c2662b416907
Step 10/22 : COPY Gemfile* ${APP_PATH}/
 ---> Using cache
 ---> 7282b1f406f0
Step 11/22 : RUN gem update --system   && gem install bundler:${BUNDLER_VERSION}   && bundle install -j4 --retry 3 --without development test  && rm -rf /usr/local/bundle/cache/*.gem   && find /usr/local/bundle/gems/ -name "*.c" -delete   && find /usr/local/bundle/gems/ -name "*.o" -delete
 ---> Running in 21011c4b87e0
Updating rubygems-update
Successfully installed rubygems-update-3.2.13
Installing RubyGems 3.2.13
  Successfully built RubyGem
  Name: bundler
  Version: 2.2.13
  File: bundler-2.2.13.gem
Bundler 2.2.13 installed
RubyGems 3.2.13 installed
Regenerating binstubs
Regenerating plugins

# 3.2.13 / 2021-03-03

## Bug fixes:

* Support non-gnu libc linux platforms. Pull request #4082 by lloeki

# 3.2.12 / 2021-03-01

#{More changelog output...}
------------------------------------------------------------------------------

RubyGems installed the following executables:
	/usr/local/bin/gem
	/usr/local/bin/bundle
	/usr/local/bin/bundler

RubyGems system software updated
Successfully installed bundler-2.1.4
1 gem installed
[DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'development test'`, and stop using this flag
The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`.
Fetching source index from https://gems.contribsys.com/
Fetching gem metadata from https://rubygems.org/............
Fetching source index from https://gems.contribsys.com/
Could not find sorbet-static-0.5.6317-universal-darwin-14 in any of the sources
The command '/bin/sh -c gem update --system   && gem install bundler:${BUNDLER_VERSION}   && bundle install -j4 --retry 3 --without development test  && rm -rf /usr/local/bundle/cache/*.gem   && find /usr/local/bundle/gems/ -name "*.c" -delete   && find /usr/local/bundle/gems/ -name "*.o" -delete' returned a non-zero code: 7

Gemfile

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.6.5'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 6.0.3'
# Use postgresql as the database for Active Record
gem 'pg', '>= 0.18', '< 2.0'
# Use Puma as the app server
gem 'puma', '~> 5.2.2'
# Use rack timeout to expire long running requests
gem 'rack-timeout'

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.4.8', require: false

# AWS
gem 'aws-sdk-s3'

gem 'money'

gem 'kaminari'

gem 'activeadmin'

# OKTA Auth for Admin pages
gem 'devise_saml_authenticatable'

gem 'devise'

gem 'uuid'

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
end

group :development do
  # Access an interactive console on exception pages or by calling 'console' anywhere in the code.
  gem 'web-console', '>= 3.3.0'
  gem 'listen', '~> 3.4'
  # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
  gem 'spring'
  gem 'spring-watcher-listen', '~> 2.0.0'
  gem 'sorbet'
  gem 'cancan'
  gem 'pundit'
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

gem 'sentry-raven'
gem 'ddtrace'
gem 'snappy'
gem 'sidekiq'
source 'https://HIDDEN:HIDDEN@gems.contribsys.com/' do
  gem 'sidekiq-pro'
end

gem 'aasm'
gem 'sorbet-runtime'
gem 'sorbet-rails'

group :test do
  gem 'shoulda-matchers'
  gem 'rspec-sidekiq'
  gem 'rails-controller-testing'
end

group :development, :test do
  gem 'brakeman', require: false
  gem 'bundler-audit', require: false
  gem 'dotenv-rails', require: 'dotenv/rails-now'
  gem 'rubocop', require: false
  gem 'rubocop-rspec'
  gem 'rubocop-rails'
  gem 'rspec-rails'
  gem 'factory_bot_rails'
  gem 'rubocop-shopify', require: false
  gem 'faker'
end

Gemfile.lock

GIT
GEM
  remote: https://rubygems.org/
  remote: https://HIDDEN:HIDDEN@gems.contribsys.com/
  specs:
    aasm (5.1.1)
      concurrent-ruby (~> 1.0)
    actioncable (6.0.3.5)
      actionpack (= 6.0.3.5)
      nio4r (~> 2.0)
      websocket-driver (>= 0.6.1)
    actionmailbox (6.0.3.5)
      actionpack (= 6.0.3.5)
      activejob (= 6.0.3.5)
      activerecord (= 6.0.3.5)
      activestorage (= 6.0.3.5)
      activesupport (= 6.0.3.5)
      mail (>= 2.7.1)
    actionmailer (6.0.3.5)
      actionpack (= 6.0.3.5)
      actionview (= 6.0.3.5)
      activejob (= 6.0.3.5)
      mail (~> 2.5, >= 2.5.4)
      rails-dom-testing (~> 2.0)
    actionpack (6.0.3.5)
      actionview (= 6.0.3.5)
      activesupport (= 6.0.3.5)
      rack (~> 2.0, >= 2.0.8)
      rack-test (>= 0.6.3)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.2.0)
    actiontext (6.0.3.5)
      actionpack (= 6.0.3.5)
      activerecord (= 6.0.3.5)
      activestorage (= 6.0.3.5)
      activesupport (= 6.0.3.5)
      nokogiri (>= 1.8.5)
    actionview (6.0.3.5)
      activesupport (= 6.0.3.5)
      builder (~> 3.1)
      erubi (~> 1.4)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.1, >= 1.2.0)
    activeadmin (2.7.0)
      arbre (~> 1.2, >= 1.2.1)
      formtastic (~> 3.1)
      formtastic_i18n (~> 0.4)
      inherited_resources (~> 1.7)
      jquery-rails (~> 4.2)
      kaminari (~> 1.0, >= 1.0.1)
      railties (>= 5.2, < 6.1)
      ransack (~> 2.1, >= 2.1.1)
      sassc-rails (~> 2.1)
      sprockets (>= 3.0, < 4.1)
    activejob (6.0.3.5)
      activesupport (= 6.0.3.5)
      globalid (>= 0.3.6)
    activemodel (6.0.3.5)
      activesupport (= 6.0.3.5)
    activerecord (6.0.3.5)
      activemodel (= 6.0.3.5)
      activesupport (= 6.0.3.5)
    activestorage (6.0.3.5)
      actionpack (= 6.0.3.5)
      activejob (= 6.0.3.5)
      activerecord (= 6.0.3.5)
      marcel (~> 0.3.1)
    activesupport (6.0.3.5)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (>= 0.7, < 2)
      minitest (~> 5.1)
      tzinfo (~> 1.1)
      zeitwerk (~> 2.2, >= 2.2.2)
    amazing_print (1.2.2)
    arbre (1.2.1)
      activesupport (>= 3.0.0)
    ast (2.4.2)
    aws-eventstream (1.1.0)
    aws-partitions (1.429.0)
    aws-sdk-core (3.112.0)
      aws-eventstream (~> 1, >= 1.0.2)
      aws-partitions (~> 1, >= 1.239.0)
      aws-sigv4 (~> 1.1)
      jmespath (~> 1.0)
    aws-sdk-kms (1.42.0)
      aws-sdk-core (~> 3, >= 3.112.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-s3 (1.89.0)
      aws-sdk-core (~> 3, >= 3.112.0)
      aws-sdk-kms (~> 1)
      aws-sigv4 (~> 1.1)
    aws-sigv4 (1.2.2)
      aws-eventstream (~> 1, >= 1.0.2)
    bcrypt (3.1.16)
    bindex (0.8.1)
    bootsnap (1.7.2)
      msgpack (~> 1.0)
    brakeman (5.0.0)
    builder (3.2.4)
    bundler-audit (0.7.0.1)
      bundler (>= 1.2.0, < 3)
      thor (>= 0.18, < 2)
    byebug (11.1.3)
    cancan (1.6.10)
    commander (4.5.2)
      highline (~> 2.0.0)
    concurrent-ruby (1.1.8)
    concurrent-ruby-ext (1.1.8)
      concurrent-ruby (= 1.1.8)
    connection_pool (2.2.3)
    crass (1.0.6)
    ddtrace (0.45.0)
      msgpack
    devise (4.7.3)
      bcrypt (~> 3.0)
      orm_adapter (~> 0.1)
      railties (>= 4.1.0)
      responders
      warden (~> 1.2.3)
    devise_saml_authenticatable (1.6.3)
      devise (> 2.0.0)
      ruby-saml (~> 1.7)
    diff-lcs (1.4.4)
    digest-crc (0.6.3)
      rake (>= 12.0.0, < 14.0.0)
    dotenv (2.7.6)
    dotenv-rails (2.7.6)
      dotenv (= 2.7.6)
      railties (>= 3.2)
    erubi (1.10.0)
    exponential-backoff (0.0.4)
    factory_bot (6.1.0)
      activesupport (>= 5.0.0)
    factory_bot_rails (6.1.0)
      factory_bot (~> 6.1.0)
      railties (>= 5.0.0)
    faker (2.16.0)
      i18n (>= 1.6, < 2)
    faraday (1.3.0)
      faraday-net_http (~> 1.0)
      multipart-post (>= 1.2, < 3)
      ruby2_keywords
    faraday-net_http (1.0.0)
    ffi (1.14.2)
    formtastic (3.1.5)
      actionpack (>= 3.2.13)
    formtastic_i18n (0.6.0)
    globalid (0.4.2)
      activesupport (>= 4.2.0)
    has_scope (0.7.2)
      actionpack (>= 4.1)
      activesupport (>= 4.1)
    highline (2.0.3)
    i18n (1.8.9)
      concurrent-ruby (~> 1.0)
    inherited_resources (1.11.0)
      actionpack (>= 5.0, < 6.1)
      has_scope (~> 0.6)
      railties (>= 5.0, < 6.1)
      responders (>= 2, < 4)
    jmespath (1.4.0)
    jquery-rails (4.4.0)
      rails-dom-testing (>= 1, < 3)
      railties (>= 4.2.0)
      thor (>= 0.14, < 2.0)
    kaminari (1.2.1)
      activesupport (>= 4.1.0)
      kaminari-actionview (= 1.2.1)
      kaminari-activerecord (= 1.2.1)
      kaminari-core (= 1.2.1)
    kaminari-actionview (1.2.1)
      actionview
      kaminari-core (= 1.2.1)
    kaminari-activerecord (1.2.1)
      activerecord
      kaminari-core (= 1.2.1)
    kaminari-core (1.2.1)
    listen (3.4.1)
      rb-fsevent (~> 0.10, >= 0.10.3)
      rb-inotify (~> 0.9, >= 0.9.10)
    little-plugger (1.1.4)
    logging (2.3.0)
      little-plugger (~> 1.1)
      multi_json (~> 1.14)
    lograge (0.11.2)
      actionpack (>= 4)
      activesupport (>= 4)
      railties (>= 4)
      request_store (~> 1.0)
    lograge-sql (1.3.1)
      activerecord (>= 4, < 7.0)
      lograge (~> 0.4)
    loofah (2.9.0)
      crass (~> 1.0.2)
      nokogiri (>= 1.5.9)
    macaddr (1.7.2)
      systemu (~> 2.6.5)
    mail (2.7.1)
      mini_mime (>= 0.1.1)
    marcel (0.3.3)
      mimemagic (~> 0.3.2)
    method_source (1.0.0)
    mimemagic (0.3.5)
    mini_mime (1.0.2)
    mini_portile2 (2.5.0)
    minitest (5.14.3)
    money (6.14.1)
      i18n (>= 0.6.4, <= 2)
    msgpack (1.4.2)
    multi_json (1.15.0)
    multipart-post (2.1.1)
    nio4r (2.5.5)
    nokogiri (1.11.1)
      mini_portile2 (~> 2.5.0)
      racc (~> 1.4)
    oj (3.11.1)
    orm_adapter (0.5.0)
    ougai (1.9.1)
      oj (~> 3.10)
    parallel (1.20.1)
    parlour (5.0.0)
      commander (~> 4.5)
      parser
      rainbow (~> 3.0)
      sorbet-runtime (>= 0.5)
    parser (3.0.0.0)
      ast (~> 2.4.1)
    pg (1.2.3)
    phobos (1.9.0)
      activesupport (>= 3.0.0)
      concurrent-ruby (>= 1.0.2)
      concurrent-ruby-ext (>= 1.0.2)
      exponential-backoff
      logging
      ruby-kafka
      thor
    polyamorous (2.3.2)
      activerecord (>= 5.2.1)
    polyfill (1.9.0)
    puma (5.2.2)
      nio4r (~> 2.0)
    pundit (2.1.0)
      activesupport (>= 3.0.0)
    racc (1.5.2)
    rack (2.2.3)
    rack-test (1.1.0)
      rack (>= 1.0, < 3)
    rack-timeout (0.6.0)
    rails (6.0.3.5)
      actioncable (= 6.0.3.5)
      actionmailbox (= 6.0.3.5)
      actionmailer (= 6.0.3.5)
      actionpack (= 6.0.3.5)
      actiontext (= 6.0.3.5)
      actionview (= 6.0.3.5)
      activejob (= 6.0.3.5)
      activemodel (= 6.0.3.5)
      activerecord (= 6.0.3.5)
      activestorage (= 6.0.3.5)
      activesupport (= 6.0.3.5)
      bundler (>= 1.3.0)
      railties (= 6.0.3.5)
      sprockets-rails (>= 2.0.0)
    rails-controller-testing (1.0.5)
      actionpack (>= 5.0.1.rc1)
      actionview (>= 5.0.1.rc1)
      activesupport (>= 5.0.1.rc1)
    rails-dom-testing (2.0.3)
      activesupport (>= 4.2.0)
      nokogiri (>= 1.6)
    rails-html-sanitizer (1.3.0)
      loofah (~> 2.3)
    railties (6.0.3.5)
      actionpack (= 6.0.3.5)
      activesupport (= 6.0.3.5)
      method_source
      rake (>= 0.8.7)
      thor (>= 0.20.3, < 2.0)
    rainbow (3.0.0)
    rake (13.0.3)
    ransack (2.3.2)
      activerecord (>= 5.2.1)
      activesupport (>= 5.2.1)
      i18n
      polyamorous (= 2.3.2)
    rb-fsevent (0.10.4)
    rb-inotify (0.10.1)
      ffi (~> 1.0)
    redis (4.2.5)
    regexp_parser (2.0.3)
    request_store (1.5.0)
      rack (>= 1.4)
    responders (3.0.1)
      actionpack (>= 5.0)
      railties (>= 5.0)
    rexml (3.2.4)
    rspec-core (3.10.1)
      rspec-support (~> 3.10.0)
    rspec-expectations (3.10.1)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.10.0)
    rspec-mocks (3.10.1)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.10.0)
    rspec-rails (4.0.2)
      actionpack (>= 4.2)
      activesupport (>= 4.2)
      railties (>= 4.2)
      rspec-core (~> 3.10)
      rspec-expectations (~> 3.10)
      rspec-mocks (~> 3.10)
      rspec-support (~> 3.10)
    rspec-sidekiq (3.1.0)
      rspec-core (~> 3.0, >= 3.0.0)
      sidekiq (>= 2.4.0)
    rspec-support (3.10.1)
    rubocop (1.8.1)
      parallel (~> 1.10)
      parser (>= 3.0.0.0)
      rainbow (>= 2.2.2, < 4.0)
      regexp_parser (>= 1.8, < 3.0)
      rexml
      rubocop-ast (>= 1.2.0, < 2.0)
      ruby-progressbar (~> 1.7)
      unicode-display_width (>= 1.4.0, < 3.0)
    rubocop-ast (1.4.1)
      parser (>= 2.7.1.5)
    rubocop-rails (2.9.1)
      activesupport (>= 4.2.0)
      rack (>= 1.1)
      rubocop (>= 0.90.0, < 2.0)
    rubocop-rspec (1.41.0)
      rubocop (>= 0.68.1)
    rubocop-shopify (1.0.7)
      rubocop (~> 1.4)
    ruby-kafka (1.3.0)
      digest-crc
    ruby-progressbar (1.11.0)
    ruby-saml (1.11.0)
      nokogiri (>= 1.5.10)
    ruby2_keywords (0.0.2)
    safe_type (1.1.1)
    sassc (2.4.0)
      ffi (~> 1.9)
    sassc-rails (2.1.2)
      railties (>= 4.0.0)
      sassc (>= 2.0)
      sprockets (> 3.0)
      sprockets-rails
      tilt
    sentry-raven (3.1.1)
      faraday (>= 1.0)
    shoulda-matchers (4.5.1)
      activesupport (>= 4.2.0)
    sidekiq (6.1.3)
      connection_pool (>= 2.2.2)
      rack (~> 2.0)
      redis (>= 4.2.0)
    sidekiq-pro (5.2.1)
      connection_pool (>= 2.2.3)
      sidekiq (>= 6.1.0)
    snappy (0.2.0)
    sorbet (0.5.6317)
      sorbet-static (= 0.5.6317)
    sorbet-coerce (0.3.0)
      polyfill (~> 1.8)
      safe_type (~> 1.1, >= 1.1.1)
      sorbet (>= 0.4.4704)
      sorbet-runtime (>= 0.4.4704)
    sorbet-rails (0.7.3)
      method_source (>= 0.9.2)
      parlour (>= 4.0.1)
      parser (>= 2.7)
      sorbet-coerce (>= 0.2.6)
      sorbet-runtime (>= 0.5)
    sorbet-runtime (0.5.6286)
    sorbet-static (0.5.6317-universal-darwin-14)
    spring (2.1.1)
    spring-watcher-listen (2.0.1)
      listen (>= 2.7, < 4.0)
      spring (>= 1.2, < 3.0)
    sprockets (4.0.2)
      concurrent-ruby (~> 1.0)
      rack (> 1, < 3)
    sprockets-rails (3.2.2)
      actionpack (>= 4.0)
      activesupport (>= 4.0)
      sprockets (>= 3.0.0)
    systemu (2.6.5)
    thor (1.1.0)
    thread_safe (0.3.6)
    tilt (2.0.10)
    tzinfo (1.2.9)
      thread_safe (~> 0.1)
    unicode-display_width (2.0.0)
    uuid (2.3.9)
      macaddr (~> 1.0)
    warden (1.2.9)
      rack (>= 2.0.9)
    web-console (4.1.0)
      actionview (>= 6.0.0)
      activemodel (>= 6.0.0)
      bindex (>= 0.4.0)
      railties (>= 6.0.0)
    websocket-driver (0.7.3)
      websocket-extensions (>= 0.1.0)
    websocket-extensions (0.1.5)
    zeitwerk (2.4.2)

PLATFORMS
  ruby

DEPENDENCIES
  aasm
  activeadmin
  aws-sdk-s3
  bootsnap (>= 1.4.8)
  brakeman
  bundler-audit
  byebug
  cancan
  ddtrace
  devise
  devise_saml_authenticatable
  dotenv-rails
  factory_bot_rails
  faker
  kaminari
  listen (~> 3.4)
  money
  pg (>= 0.18, < 2.0)
  puma (~> 5.2.2)
  pundit
  rack-timeout
  rails (~> 6.0.3)
  rails-controller-testing
  rspec-rails
  rspec-sidekiq
  rubocop
  rubocop-rails
  rubocop-rspec
  rubocop-shopify
  sawmill!
  sentry-raven
  shoulda-matchers
  sidekiq
  sidekiq-pro!
  snappy
  sorbet
  sorbet-rails
  sorbet-runtime
  spring
  spring-watcher-listen (~> 2.0.0)
  tzinfo-data
  uuid
  web-console (>= 3.3.0)

RUBY VERSION
   ruby 2.6.5p114

BUNDLED WITH
   2.1.4

Notes

This works fine when I pin the rubygems-update version to 3.2.12. Of course I'm installing onto Linux in this environment, while my Gemfile.lock specifies darwin since I'm developing on OSX locally. I'm guessing that's confusing things in the new version? Using 3.2.12, bundler correctly installs sorbet-static 0.5.6317 (x86_64-linux) instead. It might be that this is a feature and not a bug, but it makes it hard to develop on a different platform than your production one.

I will abide by the code of conduct.

[deivid-rodriguez]

Hi @garybethea, thanks for the report.

It definitely looks related to #4082. My guess is that bundler, when looking for candidates of sorbet-static for installation, is now trying to find an exact candidate for your platform (alpine), and it's no longer finding it because after #4082 it wants a linux-musl version, which doesn't exist.

Now I'm rereading the description of the PR and I think it states that this is expected:

Impact of this change is expected to be limited to none, as -linux gems working on non--linux platforms such as *-linux-musl are vanishingly rare due to the stringent requirements and moves required to produce them. Therefore, as codified in the codebase preceding this pull request, it can be assumed that the set of gems published as *-linux is strictly equal to the set of gems that would be marked -linux-gnu only (if that were a thing).

I totally missed this when considering this PR, I thought this would be backwards compatible in the sense that generic linux versions would still be chosen on alpine if there's not an alpine specific variant. I don't think assuming that non of the linux specific variants of a gem won't work on alpine. For example, I believe nokogori's linux variant has always worked fine on alpine?

@lloeki Can you have a look at this? Assuming what I explained is the culprit of this issue, and that it is expected from the implementation of #4082, I'm afraid we need a better solution to support -linux-musl variants.

[deivid-rodriguez]

Also @garybethea, I don't think it will fix this issue, but could you upgrade to bundler 2.2.13? It should have a better behaviour regarding platforms. For example, the error message on this issue ("Could not find sorbet-static-0.5.6317-universal-darwin-14 in any of the sources") should be less confusing with the latest bundler. Assuming you regenerate the lockfile and lock both platforms you're using (alpine and darwin), I believe it should at least give you a better error message saying that it couldn't find a musl specific variant.

[garybethea]

Thanks @deivid-rodriguez I'll give upgrading bundler a try.

I don't think assuming that non of the linux specific variants of a gem won't work on alpine. For example, I believe nokogori's linux variant has always worked fine on alpine?

I can confirm that on Alpine, the x86_64-linux version of sorbet-static works fine!

[deivid-rodriguez]

Yeah, I can confirm the culprit. I don't know how I missed this since it was stated clearly on the PR body and I knew linux specific variants that work just fine on alpine 😞.

The issue is clearly not good for sobet-static since it only has platform specific variants. For gems providing a generic ruby version like nokogiri I guess the best choice is more open to debate, since at least it will fallback to the ruby variant (although it will probably fail due to missing compilation tools). But since nobody has complained about linux variants on alpine for gems like nokogiri or sorbet-static, we should keep the previous behavior unless the author has explicitly published a musl variant.

[lloeki]

@deivid-rodriguez I believe you are correct, as as of the PR linux and linux-musl are different things, which means e.g sorbet-static would need to be republished as is under a linux-musl platform. Sadly I thought that was understood given the description and the tests. Let me apologise for the miscommunication.

I think sorbet-static works on all platforms because it’s (as the name says) statically built, and doesn’t call anything that behaves differently on musl than on glibc.

IIUC the proposal you’re making is:

• a gem published as linux only would get pulled by a linux-musl Ruby
• a gem published as both linux-musl and linux would get the former on a linux-musl Ruby, ignoring the latter

The counterpart is that binary gems that are built and published on linux only, especially if they depend on gnuisms or C++/non-static ones, have a good chance of being broken on musl. This was the case that I encountered (and the reason why upstream Alpine patches out binary platforms, ensuring only ruby remains) and wanted to solve.

So, IIUC a more complete solution would be about adding the following:

• a gem published as linux only but neither linux-musl nor ruby would get pulled by both a linux and a linux-musl Ruby
• a gem published as both linux-musl and linux but not ruby would get the former on a linux-musl Ruby, ignoring the latter
• a gem published as ruby and linux only but not linux-musl would get pulled as linux by a linux Ruby and as ruby by a linux-musl Ruby
• a gem published as both linux-musl and linux as well as ruby would get the former on a linux-musl Ruby, ignoring the latter, while it would get a linux gem on a linux Ruby

IOW the presence of a linux-musl gem hints at the linux gem being incompatible, while the presence of a ruby gem along a linux one but no musl gives a chance to a musl platform to have a working gem (pending proper build dependency installation), and finally a lone linux gem would mean attempt to install no matter the platform (but may still break if it depends on gnuisms).

This introduces quite an asymmetry between the published gem platform and the querying Ruby platform, as well as being much more complex to implement. This is why I went the symmetric route, which although it requires republishing some gems if there’s no ruby platform gem, would end up always working in the long run, with a temporary breakage for an already broken platform (although on different packages). Again I apologise for the miscommunication on that point as I erroneously thought this consequence was clarified enough (which it obviously was not).

So, WDYT would be the best course from here on?

[lloeki]

From an implementation point of view, I’m not sure this can be solved solely with comparisons as knowledge of availability of other gem platforms needs to be taken care of.

[lloeki]

I also had a wild crazy thought that maybe binary gems could claim to support multiple platforms, so a gem such as sorbet-static could claim support for linux and linux-musl, and similarly darwin gems could claim to support e.g darwin17 through darwin20 (a range whose start is known via -mmacos-min-version flag value) without having to publish an identical one for each.

[deivid-rodriguez]

Well, I don't think you miscommunicated anything, it was all cristal clear, I just missed it. I think I read it the first time and didn't raise any flags, and then this second time I was not careful enough to go through everything again :(.

I think we should go back to the previous state of things, with the exception that if there's a musl variant, it should always be preferred on alpine. For the cases where the linux version doesn't work on musl, and the gem author hasn't provided a working musl variant, I think the feature in #4049 should allow users to manually opt-into the generic version, so I'll commit to shipping that soon.

[garybethea]

That's amazing, thanks for the support @deivid-rodriguez!

[lloeki]

I think the feature in #4049 should allow users to manually opt-into the generic version

Oh, I was not aware that was a planned thing. IIUC that's the equivalent of this gem command flag, right?

gem install --help 
Usage: gem install GEMNAME [GEMNAME ...] [options] -- --build-flags [options]

  Options:
        --platform PLATFORM          Specify the platform of gem to install

Am I correct to assume this flag would be persisted in the lockfile in some way?

Is the information persistable/persisted per-platform? Otherwise while it would work, it would mean that ruby would be that gem platform for all Ruby platforms, thus having gems being ruby on every other platform than linux-musl (where binaries work) just because the linux gem happens to not be compatible with Alpine. It's a fair workaround, but I would surmise that some devs would not enjoy that ;)

with the exception that if there's a musl variant, it should always be preferred on alpine.

I would also add as a mirror constraint that on a linux Ruby it should never pick linux-musl gems (to be thorough, any other linux-* except linux-gnu), which before #4082 can happen theoretically if a gem is only published as linux-musl and in practice when there's a delay between pushing a linux-musl and a linux one.

From a user experience point of view this puts the burden on the developer (as opposed to the gem maintainer) to know which linux gems work on musl and which don't, but overall it would be a fair workaround and progress from the situation before I wrote #4082, and an easier transition than after 3.2.13.

@deivid-rodriguez WDYT? Should we revert #4082 and reopen #3174? Also, would you need help in implementing that while you work on #4049?

[deivid-rodriguez]

This new :force_ruby_platform flag is just a per-gem implementation of a global configuration bundler has, BUNDLE_FORCE_RUBY_PLATFORM, which completely forbids bundler from picking up platform specific gems. This would just make that a bit less brute-force.

The rubygems --platform flag is similar (more flexible I guess since it allows you to specify any platform, not just ruby), but bundler doesn't currently understand it.

Is the information persistable/persisted per-platform? Otherwise while it would work, it would mean that ruby would be that gem platform for all Ruby platforms, thus having gems being ruby on every other platform than linux-musl (where binaries work) just because the linux gem happens to not be compatible with Alpine. It's a fair workaround, but I would surmise that some devs would not enjoy that ;)

That's is a fair concern, yeah the flag is currently per gem, but not per gem+platform. An improvement over the current situation but definitely not perfect.

I would also add as a mirror constraint that on a linux Ruby it should never pick linux-musl gems (to be thorough, any other linux-* except linux-gnu), which before #4082 can happen theoretically if a gem is only published as linux-musl and in practice when there's a delay between pushing a linux-musl and a linux one.

Absolutely 👍.

From a user experience point of view this puts the burden on the developer (as opposed to the gem maintainer) to know which linux gems work on musl and which don't, but overall it would be a fair workaround and progress from the situation before I wrote #4082, and an easier transition than after 3.2.13.

Yes, I agree with you. Both on a the need for better tools for gem authors to be able to inform users about what works and what not, and on the fact that improving what has been proposed would still be a good move forward.

@deivid-rodriguez WDYT? Should we revert #4082 and reopen #3174? Also, would you need help in implementing that while you work on #4049?

I'd say, yes, and yes. I have a lot on my plate so any help is appreciated.

[lloeki]

I'd say, yes, and yes. I have a lot on my plate so any help is appreciated.

Great. Same here, but I'll do my best to give a hand.

[deivid-rodriguez]

For now I'll be reverting the change (#4434) and release the revert as 3.2.14.

[lloeki]

For example, I believe nokogori's linux variant has always worked fine on alpine?

@deivid-rodriguez well apparently it doesn't always work (my hunch is it appears to often work because it falls back to source build)

[deivid-rodriguez]

I think nokogiri expects to support alpine with their single precompiled linux gem, so the issue should probably be reported there.

[lloeki]

This issue has been torturing me for a while as I could not see how sorbet-static could work on musl.

Indeed:

$ docker run --rm -it ruby:3.0-alpine /bin/sh
# gem install sorbet-static
# find /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/
/usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/
/usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec
/usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet
# ldd /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet 
	/lib64/ld-linux-x86-64.so.2 (0x7fc9c479e000)
	libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fc9c479e000)
	libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fc9c479e000)
	libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7fc9c479e000)
	libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fc9c479e000)
	librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7fc9c479e000)
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __register_atfork: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: strtoll_l: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: strtoull_l: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __cxa_thread_atexit_impl: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __isinff: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __isinf: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __isinfl: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __isnanf: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __isnan: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __vsnprintf_chk: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: backtrace: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: backtrace_symbols: symbol not found
Error relocating /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: __fdelt_chk: symbol not found

So the following is entirely expected (the "thing" not found is /lib64/ld-linux-x86-64.so.2):

# /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet
/bin/sh: /usr/local/bundle/gems/sorbet-static-0.5.6357-x86_64-linux/libexec/sorbet: not found

Unless I'missing some critical piece (which I would be glad to hear), there's no way sorbet-static ever ran as is on a off-the-shelf musl setup.

nokogiri, though, is another beast, and works fine through dark magic:

# try both in turn:
docker run --rm -it ruby:3.0 /bin/bash
docker run --rm -it ruby:3.0-alpine /bin/bash

# then:
wget https://rubygems.org/downloads/nokogiri-1.11.2-x86_64-linux.gem
mkdir nokogiri
cd nokogiri/
tar xf ../nokogiri-1.11.2-x86_64-linux.gem 
mkdir data
cd data/
tar xzf ../data.tar.gz
ldd lib/nokogiri/3.0/nokogiri.so   # the output will be different on gnu and musl

# and so it works on both
gem install ../../nokogiri-1.11.2-x86_64-linux.gem
ruby -rnokogiri -e 'puts Nokogiri::HTML("<div id=foo>FOO</div><div id=bar>BAR</div>").css("#foo").first.content'