Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found #429

Closed
slondr opened this issue Dec 16, 2022 · 19 comments
Closed

/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found #429

slondr opened this issue Dec 16, 2022 · 19 comments

Comments

@slondr
Copy link

slondr commented Dec 16, 2022

I made sure the issue is in bootsnap

Yep (also, btw, there is a typo in the GitHub issue template which appears right after this header)

Steps to reproduce

Run any Rails command after installing bootsnap via bundler

Expected behavior

In this case: I ran rails server and expected my server to start

Actual behavior

Application crashed almost immediately, with a big error message starting with LoadError originating in bootsnap.so. I posted the full error message at the bottom of this issue.

System configuration

Bootsnap version: 1.15.0

Ruby version: 2.7.5

Rails version: 6.0.5

Full error message

Puma starting in single mode...
* Puma version: 5.6.5 (ruby 2.7.5-p203) ("Birdie's Version")
*  Min threads: 0
*  Max threads: 5
*  Environment: development
*          PID: 192898
! Unable to load application: LoadError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/bootsnap.so) - /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/bootsnap.so
Traceback (most recent call last):
	31: from /home/user/.rvm/gems/ruby-2.7.5/bin/puma:23:in `<main>'
	30: from /home/user/.rvm/gems/ruby-2.7.5/bin/puma:23:in `load'
	29: from /home/user/.rvm/gems/ruby-2.7.5/gems/puma-5.6.5/bin/puma:10:in `<top (required)>'
	28: from /home/user/.rvm/gems/ruby-2.7.5/gems/puma-5.6.5/lib/puma/cli.rb:81:in `run'
	27: from /home/user/.rvm/gems/ruby-2.7.5/gems/puma-5.6.5/lib/puma/launcher.rb:193:in `run'
	26: from /home/user/.rvm/gems/ruby-2.7.5/gems/puma-5.6.5/lib/puma/single.rb:44:in `run'
	25: from /home/user/.rvm/gems/ruby-2.7.5/gems/puma-5.6.5/lib/puma/runner.rb:150:in `load_and_bind'
	24: from /home/user/.rvm/gems/ruby-2.7.5/gems/puma-5.6.5/lib/puma/configuration.rb:270:in `app'
	23: from /home/user/.rvm/gems/ruby-2.7.5/gems/puma-5.6.5/lib/puma/configuration.rb:348:in `load_rackup'
	22: from /home/user/.rvm/gems/ruby-2.7.5/gems/rack-2.2.3.1/lib/rack/builder.rb:66:in `parse_file'
	21: from /home/user/.rvm/gems/ruby-2.7.5/gems/rack-2.2.3.1/lib/rack/builder.rb:105:in `load_file'
	20: from /home/user/.rvm/gems/ruby-2.7.5/gems/rack-2.2.3.1/lib/rack/builder.rb:116:in `new_from_string'
	19: from /home/user/.rvm/gems/ruby-2.7.5/gems/rack-2.2.3.1/lib/rack/builder.rb:116:in `eval'
	18: from config.ru:3:in `block in <main>'
	17: from /home/user/.rvm/rubies/ruby-2.7.5/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	16: from /home/user/.rvm/rubies/ruby-2.7.5/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	15: from /home/user/dev/core/config/environment.rb:2:in `<top (required)>'
	14: from /home/user/.rvm/rubies/ruby-2.7.5/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	13: from /home/user/.rvm/rubies/ruby-2.7.5/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	12: from /home/user/dev/core/config/application.rb:1:in `<top (required)>'
	11: from /home/user/.rvm/rubies/ruby-2.7.5/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	10: from /home/user/.rvm/rubies/ruby-2.7.5/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	 9: from /home/user/dev/core/config/boot.rb:4:in `<top (required)>'
	 8: from /home/user/dev/core/config/boot.rb:4:in `require'
	 7: from /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/setup.rb:5:in `<top (required)>'
	 6: from /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap.rb:100:in `default_setup'
	 5: from /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap.rb:57:in `setup'
	 4: from /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/compile_cache.rb:16:in `setup'
	 3: from /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/compile_cache.rb:16:in `require_relative'
	 2: from /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/compile_cache/iseq.rb:3:in `<top (required)>'
	 1: from /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
/home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require': /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/bootsnap.so) - /home/user/.rvm/gems/ruby-2.7.5/gems/bootsnap-1.15.0/lib/bootsnap/bootsnap.so (LoadError)
@casperisfine
Copy link
Contributor

Interesting. Likely caused by 0de36ee, but I wonder if that's even needed.

@gabrielrinaldi
Copy link

I am also having the same issue, on my Mac it runs fine, but when dockerizing it it fails with the same message. For some reason building the docker image on my Mac does not cause the issue, but it does on the CI machine.

@gabrielrinaldi
Copy link

After more digging I think this is related to the new release of the images, which only has GLIBC_2.35, would that make sense?

@casperisfine
Copy link
Contributor

Yeah that would male sense. if the base image is public would be nice to share it, that would help confirm the bug on my side.

That said I really don't see why we're put the whole glibc in the cache key. I'm pretty sure we can just remove that code, but I need to dig in the history to confirm.

@casperisfine
Copy link
Contributor

Hum, wait, something super weird here. We remove the GLIBC version check in 1.12.0 https://github.com/Shopify/bootsnap/blob/main/CHANGELOG.md#1120

@casperisfine
Copy link
Contributor

Do you re-use your bundler cache across docker image builds? I'm thinking you might need to recompile bootsnap.

@gabrielrinaldi
Copy link

gabrielrinaldi commented Dec 19, 2022

This is my docker file:

# Base image
ARG RUBY_VERSION=3.1.3
ARG VARIANT=slim-bullseye

FROM ruby:${RUBY_VERSION}-${VARIANT} as base

ARG BUNDLE_PATH=vendor/bundle
ARG BUNDLE_WITHOUT=development:test
ARG RAILS_ENV=production

ENV BUNDLE_PATH ${BUNDLE_PATH}
ENV BUNDLE_WITHOUT ${BUNDLE_WITHOUT}
ENV RAILS_ENV=${RAILS_ENV}

RUN mkdir /app

WORKDIR /app

RUN mkdir -p tmp/pids

# Build dependencies
FROM base as build_deps

ARG DEV_PACKAGES="git build-essential libpq-dev wget vim curl gzip xz-utils"

ENV DEV_PACKAGES ${DEV_PACKAGES}

RUN --mount=type=cache,id=dev-apt-cache,sharing=locked,target=/var/cache/apt \
  --mount=type=cache,id=dev-apt-lib,sharing=locked,target=/var/lib/apt \
  apt-get update -qq && \
  apt-get install --no-install-recommends -y ${DEV_PACKAGES} \
  && rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Build gems
FROM build_deps as gems

ARG BUNDLER_VERSION=2.3.26

ENV BUNDLER_VERSION ${BUNDLER_VERSION}

RUN gem install -N bundler -v ${BUNDLER_VERSION}

COPY Gemfile* ./
RUN bundle install && rm -rf vendor/bundle/ruby/*/cache

# Production image
FROM base

ARG NODE_VERSION=18.12.1
ARG PROD_PACKAGES="postgresql-client file vim curl gzip gnupg ca-certificates wget xz-utils"
ARG SERVER_COMMAND="bin/rails fly:server"

ENV PORT 8080
ENV PROD_PACKAGES=${PROD_PACKAGES}
ENV SERVER_COMMAND ${SERVER_COMMAND}

RUN groupadd --gid 1000 node \
  && useradd --uid 1000 --gid node --shell /bin/bash --create-home node

RUN --mount=type=cache,id=prod-apt-cache,sharing=locked,target=/var/cache/apt \
  --mount=type=cache,id=prod-apt-lib,sharing=locked,target=/var/lib/apt \
  apt-get update -qq && \
  apt-get install --no-install-recommends -y \
  ${PROD_PACKAGES} \
  && rm -rf /var/lib/apt/lists /var/cache/apt/archives

RUN set -ex \
  && for key in \
  4ED778F539E3634C779C87C6D7062848A1AB005C \
  141F07595B7B3FFE74309A937405533BE57C7D57 \
  74F12602B6F1C4E913FAA37AD3A89613643B6201 \
  61FC681DFB92A079F1685E77973F295594EC4689 \
  8FCCA13FEF1D0C2E91008E09770F7A9A5AE15600 \
  C4F0DFFF4E8C1A8236409D08E73BC641CC11F4C8 \
  890C08DB8579162FEE0DF9DB8BEAB4DFCF555EF4 \
  C82FA3AE1CBEDC6BE46B9360C43CEC45C17AB93C \
  108F52B48DB57BB0CC439B2997B01419BD92F80A \
  ; do \
  gpg --keyserver hkps://keys.openpgp.org --recv-keys "$key"; \
  done

RUN ARCH= && dpkgArch="$(dpkg --print-architecture)" \
  && case "${dpkgArch##*-}" in \
  amd64) ARCH='x64';; \
  ppc64el) ARCH='ppc64le';; \
  s390x) ARCH='s390x';; \
  arm64) ARCH='arm64';; \
  armhf) ARCH='armv7l';; \
  i386) ARCH='x86';; \
  *) echo "unsupported architecture"; exit 1 ;; \
  esac \
  && set -x \
  && curl -fsSLO --compressed "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-$ARCH.tar.xz" \
  && curl -fsSLO --compressed "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" \
  && gpg --batch --decrypt --output SHASUMS256.txt SHASUMS256.txt.asc \
  && grep " node-v$NODE_VERSION-linux-$ARCH.tar.xz\$" SHASUMS256.txt | sha256sum -c - \
  && tar -xJf "node-v$NODE_VERSION-linux-$ARCH.tar.xz" -C /usr/local --strip-components=1 --no-same-owner \
  && rm "node-v$NODE_VERSION-linux-$ARCH.tar.xz" SHASUMS256.txt.asc SHASUMS256.txt \
  && apt-get purge -y --auto-remove xz-utils \
  && ln -s /usr/local/bin/node /usr/local/bin/nodejs

RUN npm install -g mjml

COPY --from=gems /app /app

ENV SECRET_KEY_BASE 1

COPY . .

RUN bin/rails fly:build

EXPOSE ${PORT}

CMD ${SERVER_COMMAND}

fly:build is just an alias to assets:precompile

@casperisfine
Copy link
Contributor

Ok, so I suspect the issue might come from the fact that you compile the gems in a container, then import in your image.

I assume build-essential upgrade the libc or something.

Just for debugging purposes can you try running that same apt-get install in the main image? If that fix the issue it will confirm that it's the problem. And then the proper fix might be to upgrade only the libc package to make sure it match the one used for compilation.

@gabrielrinaldi
Copy link

gabrielrinaldi commented Dec 22, 2022

Removing the caching does solve the problems (everything in one image), but adding build-essential to the final image does not make a difference.

I think it is fine to close this as a bootsnap related issue, the issue seems to be ubuntu-latest becoming 22 vs 20. It just coincided for me that both happened at the same time (bootsnap upgrade and the ubuntu-latest).

@slondr
Copy link
Author

slondr commented Jan 3, 2023

Can this issue be re-opened? gabrielrinaldi's problem seems unrelated to mine — I am not using containers or macOS.

@casperisfine
Copy link
Contributor

The issue is likely to be similar nonetheless. Please try to flush your bundler cache (or just uninstall bootsnap and re-bundle).

If the issue persists I'll re-open, but I'll need more info to investigate.

@tcarac
Copy link

tcarac commented Jan 12, 2023

I'm observing a similar issue after having upgraded to ruby 3.2

/home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/setup.rb:5:in <top (required)>'
/home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap.rb:100:in default_setup' /home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap.rb:57:in setup'
/home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/compile_cache.rb:16:in setup' /home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/compile_cache.rb:16:in require_relative'
/home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/compile_cache/iseq.rb:3:in <top (required)>' /home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in require'
/home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in require' LoadError: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.33' not found (required by /home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/bootsnap.so) - /home/deployer/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/bootsnap.so`

@slondr
Copy link
Author

slondr commented Jan 18, 2023

@casperisfine Uninstalling bootsnap, re-bundling, then reinstalling bootsnap did not fix the issue.

What other info do you need from me?

@casperisfine
Copy link
Contributor

What other info do you need from me?

At this point, I'm at a loss as to what it might be. If you could repro this in isolation via a Dockerfile that would help a lot.

In the meantime I'll ask some colleagues if that rings a bell.

@casperisfine casperisfine reopened this Jan 18, 2023
@XrXr
Copy link
Contributor

XrXr commented Jan 18, 2023

Does ruby -e 'require "bootsnap"; require "bootsnap/bootsnap"' give you the same error? If so, something is off with the native extension. Maybe there are some hints in the output from MAKE='make V=1' gem install bootsnap --verbose --version 1.15.0 and ruby -rrbconfig -e 'pp RbConfig::CONFIG'.

@joemsak
Copy link

joemsak commented Mar 31, 2023

I had the same issue with my docker container and ruby 3.2.1

But I confirmed for me it was not bootsnap, by commenting out the line that required it, the same error just moved on to another gem

So for me the problem was because I had bundle path set to vendor/bundle and had ran "bundle update" from my host machine. Removing vendor/bundle and forcing our setup scripts to run bundle install inside the container solved it for me.

They key point I found in my google search was some linux forums result for the same issue on another library, someone had said the library had compiled with a different version of glibc. That made me realize I had installed my gems against my host machine's glibc and not the container's

I hope this helps someone else and sorry if it is not relevant to OP's situation

@casperisfine
Copy link
Contributor

Thanks for the info. I'll close then.

@eggplants
Copy link

I faced this problem when I try to run rspec test inside of the container on GitHub Actions...

<internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require': /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/bootsnap.so) - /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/bootsnap.so (LoadError)
	from <internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/compile_cache/iseq.rb:3:in `<top (required)>'
	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/compile_cache.rb:16:in `require_relative'
	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/compile_cache.rb:16:in `setup'
	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap.rb:57:in `setup'
	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap.rb:100:in `default_setup'
	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/setup.rb:5:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
	from <internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
	from /app/config/boot.rb:5:in `<top (required)>'
	from /app/config/application.rb:1:in `require_relative'
	from /app/config/application.rb:1:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
	from <internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
	from /app/vendor/bundle/ruby/3.0.0/gems/spring-4.1.1/lib/spring/application.rb:92:in `preload'
	from /app/vendor/bundle/ruby/3.0.0/gems/spring-4.1.1/lib/spring/application.rb:166:in `serve'
	from /app/vendor/bundle/ruby/3.0.0/gems/spring-4.1.1/lib/spring/application.rb:148:in `block in run'
	from /app/vendor/bundle/ruby/3.0.0/gems/spring-4.1.1/lib/spring/application.rb:142:in `loop'
	from /app/vendor/bundle/ruby/3.0.0/gems/spring-4.1.1/lib/spring/application.rb:142:in `run'
	from /app/vendor/bundle/ruby/3.0.0/gems/spring-4.1.1/lib/spring/application/boot.rb:19:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/site_ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from -e:1:in `<main>'
Error: Process completed with exit code 1.

@eggplants
Copy link

eggplants commented Jun 23, 2023

Anyway, this problem was resolved with:

diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
index b872ba633..5c0dfd1b4 100644
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -24,12 +24,14 @@ jobs:
       - uses: actions/checkout@v3

       # Install Gems with Bundler
-      - uses: ruby/setup-ruby@v1
-        with:
-          bundler-cache: true
-      - run: |
-          bundle config path vendor/bundle
-          bundle install --jobs 4
+      # https://github.com/Shopify/bootsnap/issues/429#issuecomment-1491988620
+      # - uses: ruby/setup-ruby@v1
+      #   with:
+      #     bundler-cache: true
+      # - run: |
+      #     bundle config path vendor/bundle
+      #     bundle install --jobs 4

       # Install Docker Layers
       - id: buildx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants