Locking to a specific bundler version

[jdelStrother]

It seems like we can only specify Bundler 1 or 2. Any chance of being able to supply a more specific version? We're having some issues with Bundler 2.2 (rubyjs/libv8#310) so want to lock to 2.1.4 for now

[dentarg]

From rubygems/rubygems#4123 (comment) it sounds like rubygems / bundler itself should take care of using the correct Bundler version (needs Ruby version > 2.5 though). What version of Ruby are you using @jdelStrother? Any public build you can share?

[deivid-rodriguez]

Yes, that is correct although currently it only works if multiple versions are installed. Say, if your lock file has BUNDLED WITH 2.1.4 and both 2.1.4 and 2.2.0 are installed, then 2.1.4 will be chosen. However, if there's only a single version available, we'll default to that version.

I think what's happening in this case is that ruby/setup-ruby will install the latest version, not the exact version, so that version will get run since it's the only one installed.

As a workaround, you can specifiy bundler: none and then install the proper version yourself by parsing the version from the lock file.

gem install bundler -v $(ruby -e 'puts File.read("Gemfile.lock").split("BUNDLED WITH").last.strip')

[deivid-rodriguez]

Also, for completeness, we had decided to remove the version switching logic from bundler/rubygems and I was planning to get to that after the 2.2.0 release, but this unintended incompatibilities make me hesitate a lot.

My current proposal for rubygems/bundler would be that bundle install automatically picks up the BUNDLED WITH version, installs that, and then installs the rest of the bundle using the BUNDLED WITH version. Then at that point bundle exec would always succeed and use the version specified in the BUNDLED WITH section of the lockfile.

Regardless of what we end up doing in bundler/rubygems,maybe ruby/setup-ruby could provide

bundler: none
bundler: exact
bundler: latest

so that users can customize this behaviour?

[deivid-rodriguez]

Actually there's bundler: Gemfile.lock that maybe does this already? 🤔

[jdelStrother]

It looks like it just uses the major version (ie 1 or 2) from the lockfile :

setup-ruby/index.js

Lines 175 to 191 in 084885e

	function readBundledWithFromGemfileLock(lockFile) {
	if (lockFile !== null && fs.existsSync(lockFile)) {
	const contents = fs.readFileSync(lockFile, 'utf8')
	const lines = contents.split(/\r?\n/)
	const bundledWithLine = lines.findIndex(line => /^BUNDLED WITH$/.test(line.trim()))
	if (bundledWithLine !== -1) {
	const nextLine = lines[bundledWithLine+1]
	if (nextLine && /^\d+/.test(nextLine.trim())) {
	const bundlerVersion = nextLine.trim()
	const majorVersion = bundlerVersion.match(/^\d+/)[0]
	console.log(`Using Bundler ${majorVersion} from ${lockFile} BUNDLED WITH ${bundlerVersion}`)
	return majorVersion
	}
	}
	}
	return null
	}

[eregon]

So far ruby/setup-ruby expects all Bundler 1.x releases, and Bundler 2.x releases, to be compatible essentially (modulo Bundler bugs which should be fixed in Bundler).
If we by default pin to some Bundler version, then we'll just delay findings Bundler issues in new releases.

That said I was wondering recently for Ruby 2.7+ which ship Bundler 2 whether we should use the default Bundler gem and not try to gem install bundler -v '~> 2'.
This is already done for -head versions, because those are more likely to break with Bundler releases (e.g., if there were changes directly in ruby/ruby's lib/bundler).

Agreed on being able to say bundler: 2.1.4 (which seems to already work but is not well tested).

Not sure regarding Gemfile.lock, Bundler will already pick the same major version, so it won't guarantee the same exact versions. And most gems don't use a Gemfile.lock, so this wouldn't help most cases I think.
But not really against it.
This won't work for -head versions though.

Also, for completeness, we had decided to remove the version switching logic from bundler/rubygems and I was planning to get to that after the 2.2.0 release, but this unintended incompatibilities make me hesitate a lot.

Bugs always happen, now it's important to fix them. If people never try the new releases, then there will be other issues, possibly discovered very late. In any case, the version switching logic currently accept same major, so it doesn't solve anything here.

[eregon]

My current proposal for rubygems/bundler would be that bundle install automatically picks up the BUNDLED WITH version, installs that, and then installs the rest of the bundle using the BUNDLED WITH version. Then at that point bundle exec would always succeed and use the version specified in the BUNDLED WITH section of the lockfile.

Might be a good idea, although again that only works with an existing Gemfile.lock.

FWIW I'm not happy to need so much code in ruby/setup-ruby so Bundler actually works and to try to pick the "right" Bundler.
IMHO, latest Bundler should work for all cases, except no-longer supported Ruby versions for which we'd force Bundler 1, always.

[eregon]

On Ruby < 2.7, there is no Bundler 2 included, so I think the only reasonable default (no Gemfile.lock, no explicit bundler: ...) is to pick latest Bundler.

So yeah, choosing a specific Bundler release seems the only solution for Ruby < 2.7. For Ruby 2.7+ we could choose to pick the Bundler shipped with it if it's the same major version, like we do for -head Ruby versions.

[eregon]

PR to allow using an exact version with the bundler: input, and to use the exact version if set in the lockfile: #127
@jdelStrother Could you try it with uses: eregon/setup-ruby@exact-bundler-version ?

[dentarg]

Agreed on being able to say bundler: 2.1.4 (which seems to already work but is not well tested).

I can confirm it works already and that was good, because for some reason I haven't found yet, our Actions builds started to takes ~50 minutes to run setup-ruby (with bundler-cache: true) when Bundler 2.2.0 started to be used. Hopefully I can reproduce it on a public repo somewhere.

[deivid-rodriguez]

Bugs always happen, now it's important to fix them. If people never try the new releases, then there will be other issues, possibly discovered very late. In any case, the version switching logic currently accept same major, so it doesn't solve anything here.

I know we have already discussed this in the past, and we disagree here. In my opinion they are two different valid approaches, call it discoverability vs determinism. I want to fix bugs for people who are suffering from them, and to ship features to people that need them. But I don't want to unexpectedly get in the middle at all for people for whom bundler is working just fine. By making sure a known working version of bundler is used if we can, I expect to not get in the middle suddenly when we release new version. People might experience issues, yeah, but when they choose to upgrade, not when we choose to release.

Might be a good idea, although again that only works with an existing Gemfile.lock.

Of course, bundler auto managing its own version would only apply when there's an existing lockfile. Otherwise we'd run the latest as usual. If we implement this, I expect you, and heroku, to be able to eventually remove their own logic for finding "the right bundler".

[eregon]

our Actions builds started to takes ~50 minutes to run setup-ruby (with bundler-cache: true) when Bundler 2.2.0 started to be used.

Most likely because it started compiling some gem from source instead of using a precompiled binary. google-protobuf (and grpc depends on it) is a typical example of a gem with precompiled binaries even on Linux & macOS.

[eregon]

Supported and released in https://github.com/ruby/setup-ruby/releases/tag/v1.59.0

[deivid-rodriguez]

Yes, the ~50 minutes thing sounds like what @eregon says, and it's an issue I think we've introduced in the latest bundler. We expected it to choose precompiled gems more smartly, and it seems to be doing the oppossite and not using the version with the precompiled binaries. 😞

I'm currently investigating this thing. I'll get back to it on Monday.

[deivid-rodriguez]

Thanks so much for that @eregon! That's super helpful for me ❤️

[dentarg]

@deivid-rodriguez is Bundler compiling when the output says Resolving dependencies?

I got the following output from setup-ruby when I cancelled a workflow that had been running for ~30 minutes:

bundle install
  /opt/hostedtoolcache/Ruby/2.7.2/x64/bin/bundle config --local path vendor/bundle
  /opt/hostedtoolcache/Ruby/2.7.2/x64/bin/bundle lock
  Fetching https://github.com/foo/bar.git
  Fetching gem metadata from https://rubygems.org/..........
  Fetching gem metadata from https://rubygems.org/.
  Resolving dependencies....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
  Error: The operation was canceled.

[eregon]

I believe it doesn't.
But maybe GitHub Actions is truncating the output or so, i.e., cancels/timeouts don't even send a signal, so you could have output buffered that's just not shown.
Could also be a network issue of course, which causes resolution to be so slow (but seems surprising).

[deivid-rodriguez]

Nope, it sounds like a really slow resolution happening on the new version :/. Can you share the Gemfile?

[dentarg]

@deivid-rodriguez Not really, it is a private project, using private gems (using private GitHub repos). I'll try to reproduce it somehow.

[deivid-rodriguez]

@dentarg I created a PR that I think should fix this. Can you try rubygems/rubygems#4134? Thanks!!

[dentarg]

@deivid-rodriguez Yes, rubygems/rubygems#4134 seems to solve the issue! bundle install took the more normal amount of 1m 35s using that.

I tested it using docker, details below

(The repo does not have a Gemfile.lock)

Using Bundler 2.2.0

$ docker run --rm -it -v $(pwd):/app -w /app ruby:2.7.2 bash

# ruby --version ; gem --version ; bundler --version ; gem list bundler
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]
3.1.4
Bundler version 2.1.4

*** LOCAL GEMS ***

bundler (default: 2.1.4)

# gem install bundler
Fetching bundler-2.2.0.gem
Successfully installed bundler-2.2.0
1 gem installed

# ruby --version ; gem --version ; bundler --version ; gem list bundler
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]
3.1.4
Bundler version 2.2.0

*** LOCAL GEMS ***

bundler (2.2.0, default: 2.1.4)

# bundle install
...
Resolving dependencies ...

Using rubygems/rubygems#4134

$ cd ~/src && git clone https://github.com/rubygems/rubygems.git
$ cd ~/src/rubygems
$ gh pr checkout 4134
$ docker run --rm -it -v ~/src/rubygems:/rubygems -v $(pwd):/app -w /app ruby:2.7.2 bash

# gem list bundler

*** LOCAL GEMS ***

bundler (default: 2.1.4)

# cd /rubygems
# apt-get update && apt-get install -y zip
# rake install

# gem list bundler

*** LOCAL GEMS ***

bundler (default: 2.2.0)

# cd /app
# bundle install
...
Bundle complete! 13 Gemfile dependencies, 90 gems now installed.

(GitHub authentication business left out from the above.)

[deivid-rodriguez]

Wow,that's awesome, thanks so much for verifying this!! I'll ship this fixes ASAP.

[nesrual]

I can verify that rubygems/rubygems#4134 indeed solves the problem. 👍

[deivid-rodriguez]

Thanks for verifying that @nesrual, it makes me happy :)