Preserve BUNDLE_GEMFILE and add a test for it #1893
Conversation
Starting your server with BUNDLE_GEMFILE=Gemfile.rails6 would delete this environment variable on app restart when using `--prune-bundler`, when all you really want is to restore the variable to what it was before Bundler was activated. I mimicked the existing code that preserves GEM_HOME. Co-authored-by: James Miller <bensie@gmail.com>
Test that the environment is not lost when doing a restart by using the PATH env var as a canary. While we actually care about the BUNDLER_GEMFILE env var, that one is harder to set up in testing since it requires a full copy of Puma's Gemfile. This change also has the benefit of not only protecting the BUNDLE_GEMFILE var, but also the other variables that Bundler knows about.
seven1m
changed the title
|
Hm, I wonder if we'll need to major version bump for this as people may have been relying on the old behavior of Bundle env variables getting wiped clean. |
|
It's hard to imagine the scenario where you'd depend on that behavior, since there's no way to inject the Bundler variables you do want during the Puma reload and you're left with inconsistent state depending on whether you But yeah it's up to you, I guess sometimes a bug fix is technically breaking 🤷♂ |
|
I think this is good to go, but I'm going to wait on merging it until it becomes more clear to me what the next 2-3 months of release schedule looks like. If required immediately, I suggest using a fork until that time. |
|
thank you a lot, i will test this PR when merge into master. |
|
OK, problem: for me with Bundler 1.17.2, this test passes with the original implementation. I tried a couple of other env variables, everything is passing. |
|
Ahhh! You're completely right. I don't even know what happened, but I really thought I did this right. I know this one-line change fixes issues in our applications -- we use this patch in production to allow us to reload our apps while keeping our environment variables from getting wiped out. I really struggled to write a good test to prove this, and I guess I didn't even succeed in writing a proper test. 🤦♂ I'm open to suggestions! If you're ok with it, can we leave this open and I'll work on fixing the test...? |
|
Oh of course we can leave it open. If it fixes something then the state is changed somehow, we just need to document what that is with a test. |
|
@nateberkopec @seven1m I think the test need to test with an
(In Bundler 2.1.0 that method has been renamed to |
Pretty sure I tried that and it still passed with original implementation, but now I can't remember. |
- Cannot boot puma workers from puma.log on AWS EB -- === puma startup: 2019-12-09 12:47:04 +0000 === === puma startup: 2019-12-09 12:47:04 +0000 === [19558] Early termination of worker [19600] Early termination of worker [19619] Early termination of worker [19635] Early termination of worker [19640] Early termination of worker [19646] Early termination of worker [19654] Early termination of worker [19659] Early termination of worker puma/puma#2018 puma/puma#1893
|
Thanks for the tip, I'll take a look at this and see what I can do today. |
Starting your server with BUNDLE_GEMFILE=Gemfile.rails6 would delete this environment variable on app restart when using --prune-bundler, when all you really want is to restore the variable to what it was before Bundler was activated. Writing a test for this proved to be difficult because the bug only seems to manifest itself when running the server under the following conditions: - Using `bundle exec` to run the puma command - Using a non-zero number of workers, e.g. `-w 2` - Setting `BUNDLE_GEMFILE` to something other than "Gemfile"
|
Update: I got all the builds to pass, but I'm nervous it's the wrong answer. 😬 I'm still working on it... |
|
🍿 @nateberkopec if the outstanding issue is really just Ruby 2.2-specific and you're thinking this PR can only be merged for Puma 5.x, any shot of removing support for 2.2 then? As of March 2018, 2.2 is EOL. |
|
We base Ruby version support on usage, not on EOL, so there’s a bit of lag there. |
...and go back to using Bundler.with_clean_env.
seven1m
changed the title
| @@ -297,8 +297,10 @@ def prune_bundler | |||
|
|
|||
| log '* Pruning Bundler environment' | |||
| home = ENV['GEM_HOME'] | |||
| bundle_gemfile = ENV['BUNDLE_GEMFILE'] | |||
There was a problem hiding this comment.
Something about the environment in the Travis build was causing Bundler.with_original_env to make Puma load in an endless loop. Puma would load, it would call Bundler.with_original_env, then call Kernel.exec('puma-wild', ...), then the loop would start over again.
You can see that loop in this failed build.
I could not reproduce this issue locally, even with the same version of Ubuntu, same version of Ruby, same version of Bundler, and same version of RVM. There are probably other pieces in the environment on the Travis build that I am not aware of and did not fully replicate.
Testing this required, as you can see, pushing many commits to track down the problem.
The prune_bundler method is recursive, as you know, and it relies on the line return unless defined?(Bundler) as its terminal case. Something about the Travis environment (I suspect it's RVM) was causing Bundler to be loaded every time the method recursed.
As I am out of time to work on this, I went back to the hacky solution that copies the needed env var (BUNDLE_GEMFILE) back into the environment after we run Bundler.with_clean_env.
It's not my favorite solution, at all. But it meets our needs and should not break anything in other people's environments.
There was a problem hiding this comment.
I could not reproduce this issue locally, even with the same version of Ubuntu, same version of Ruby, same version of Bundler, and same version of RVM. There are probably other pieces in the environment on the Travis build that I am not aware of and did not fully replicate.
Travis is pretty dirty in my experience, so I've switched my repos to other CIs. You can consider something similar too.
There was a problem hiding this comment.
We've moved almost everything to Github Actions, but Ruby 2.2 can't be moved (yet?).
There was a problem hiding this comment.
Ruby 2.2 is no more maintained, even security, but I understand that you want to support its users. But I personally consider its usage dangerous and prefer to drop support from gems.
There was a problem hiding this comment.
Since https://github.com/ruby/setup-ruby/releases/tag/v1.14.0 it is possible to use Ruby 2.2 on GitHub Actions... but it could be that there's something else missing for the Puma test setup (e.g. that version wasn't built with the OpenSSL version we want to test Puma with, I'm sure @MSP-Greg knows about this and maybe even have tried it, I don't have the time to double check it now)
There was a problem hiding this comment.
I should note that I believe Travis' environment actually saved us from shipping a bug here. Something in that particular environment was causing the Bundler constant to be loaded earlier than anticipated. I suspect it is some RVM magic.
There was a problem hiding this comment.
One difference between Travis and GitHub Actions is (was) the bundle install command
Travis does the following (https://travis-ci.org/puma/puma/jobs/656438353#L885)
bundle install --jobs=3 --retry=3 --path=${BUNDLE_PATH:-vendor/bundle}
Before #2126 (5 days ago), --path wasn't used on GitHub Actions, now it is:
puma/.github/workflows/ruby.yml
Lines 38 to 39 in 8f28220
Haven't checked if it matters for this particular discussion but thought it was worth mentioning here
|
Gonna test this again locally against master, but from the code perspective it LGTM. |
|
Fails on master! One more change to the changelog and then we hit the big green button. |
Starting your server with BUNDLE_GEMFILE=Gemfile.rails6 would delete this environment variable on app restart when using
--prune-bundler, when all you really want is to restore the variable to what it was before Bundler was activated.I worked with @bensie on this and this PR should supersede #1794.