Restarting via SIGUSR2 broken in 3.9.0 works in 3.8.2

[dguettler]

Steps to reproduce

1. Updated application from puma 3.8.2 to 3.9.0

2. start puma server bundle exec puma -p 3000

3. send USR2 kill -SIGUSR2 <pid>

Expected behavior

Puma server should be restarting and does so correctly in 3.8.2

Actual behavior

With version 3.9.0 the outcome is

* Listening on tcp://0.0.0.0:3000
Use Ctrl-C to stop
* Restarting...
/usr/local/lib/ruby/2.3.0/rubygems/dependency.rb:319:in `to_specs': Could not find 'puma' (>= 0.a) among 19 total gem(s) (Gem::LoadError)

System configuration

Ruby version: ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
Rails version: rails-5.0.3

[nateberkopec]

pinging @grosser who recently touched this

[grosser]

hmm I think this is because it restarts with an environment that does not have bundler
so should tweak the restart command to include bundle exec ... I though I tried that ... for now might be able to fix it by setting --restart-command manually will take a look shortly

[grosser]

it works when running via rails s which hid the error ...
@restart_argv is "/rbenv/versions/2.3.3/bin/ruby", "ruby/2.3.0/bin/puma", "-p", "3000", {:close_others=>true, 14=>14}] ... so no mention of bundler ... so it won't load bundler ... so it cannot find puma ...

the whole @restart_command is basically wrong and bundler used to override it by accident because it set a bunch of stuff in the ENV that was then inherited ... good times ...

[grosser]

fixing here #1309

[nateberkopec]

@grosser how could I repro this with a rails app? rails s, bin/puma, bundle exec puma, puma all work for me.

[grosser]

bundle exec puma did not work for me

[grosser]

the trick is to gem uninstall puma -a before doing it and having puma only installed via bundle install --path vendor/bundle

[nateberkopec]

Still nothing.

[grosser]

add gemspec to Gemfile and remove the dependencies like #1313 does then

gem uninstall puma -a
bundle exec puma test/rackup/hello.ru
ps -ef | grep puma
kill -SIGUSR2 <puma-pid>

-> kaboom

[grosser]

should be fixed in 3.9.1 too ... but it removes bundler reset functionality ... so the restart will have the old Gemfile loaded as 3.8 did ... will work on that in the meantime for next minor hopefully

[grosser]

@dguettler confirmed fixed ?

[dguettler]

@grosser 3.9.1 still seems to be failing for me

Puma starting in single mode...
* Version 3.9.1 (ruby 2.3.1-p112), codename: Private Caller
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://0.0.0.0:3000
Use Ctrl-C to stop
* Restarting...
Puma starting in single mode...
* Version 3.7.0 (ruby 2.3.1-p112), codename: Snowy Sagebrush
* Min threads: 5, max threads: 5
* Environment: development
! Unable to load application: Gem::LoadError: You have already activated puma 3.7.0, but your Gemfile requires puma 3.9.1. Prepending `bundle exec` to your command may solve this.

It tries to use globally installed version.

When uninstalling global versions I'm getting

Puma starting in single mode...
* Version 3.9.1 (ruby 2.3.1-p112), codename: Private Caller
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://0.0.0.0:3000
Use Ctrl-C to stop
* Restarting...
/Users/dguettler/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/rubygems/dependency.rb:319:in `to_specs': Could not find 'puma' (>= 0.a) among 142 total gem(s) (Gem::LoadError)

[grosser]

what bundler version are you on ?

[grosser]

nvm ... was able to reproduce ... weird ... will take another look

[grosser]

this would only happen if bundler has finally fixed the issue that bundle exec ruby -e 'puts Bundler::ORIGINAL_ENV' still included bundler variables ... that still prints all the vars locally but if I run bundle exec puma then Bundler::ORIGINAL_ENV seems to be clean ...

[grosser]

#1317 should do the trick then ...

temp-fix would be to add --restart-cmd "bundle exec puma"

[jaybloke]

Experienced the same problem - simply updating to the latest version of bundler (1.15.1) fixed it for me.

[grosser]

FYI you'd be restarting with the same Gemfile/Load-paths so changing a gem (especially puma) would not work ... use https://github.com/zendesk/samson/blob/master/config/puma.rb#L15-L33 until rubygems/bundler#5701 is released.

[bughit]

@grosser

FYI you'd be restarting with the same Gemfile/Load-paths

That does still seem to be the case, but why? BUNDLE_GEMFILE is no longer present in the new puma process but the $LOAD_PATH is still wrong/old

[grosser]

Bundle.original_env needs a patch that is in bundler master to work as intended ... for the time being the patch I linked should work fine.

…

On Mon, Jun 26, 2017 at 10:10 PM, bughit ***@***.***> wrote: @grosser <https://github.com/grosser> FYI you'd be restarting with the same Gemfile/Load-paths That does still seem to be the case, but why? BUNDLE_GEMFILE is no longer present in the new puma process but the $LOAD_PATH is still wrong/old — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAsZyVBAJalKb6tbsFWjMEaHhbgxboAks5sIBApgaJpZM4NtbSQ> .

[stereobooster]

As per this issue #1034

Yes, You should start puma without bundle exec.

But I suppose @grosser is strongly recommend to use it with bundler. What is the proper way? And why?

[grosser]

idk why anyone would want to start a random version of puma ... so I'm doing bundle exec puma ... with the ENV reset it can restart with a new Gemfile and even a new puma version.

[stereobooster]

Experiments with restart https://github.com/stereobooster/ruby-server-experiment/tree/master/puma-symlink

[grosser]

Good stuff :) Using https://github.com/zendesk/samson/blob/master/config/puma.rb#L28-L33 and everything works ... new gem/ENV on every reload ...

…

On Sun, Jul 2, 2017 at 7:54 PM, stereobooster ***@***.***> wrote: Experiments with restart https://github.com/stereobooster/ruby-server- experiment/tree/master/puma-symlink — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAsZxfstvzwsYOsJzYVZTMzGPgQHEbwks5sJ9lSgaJpZM4NtbSQ> .

[stereobooster]

@grosser https://github.com/stereobooster/ruby-server-experiment/tree/master/puma-symlink-grosser

UPD: @grosser why do you prefer USR2 over USR1 restart?

PS simplified patch

require 'bundler/setup'
on_restart do
  ENV.replace(Bundler.clean_env)
end

[glebtv]

none of the options and workarounds worked for me on 3.9.1, had to rollback to 3.8.2 which works fine

[stereobooster]

@glebtv

ruby -v
gem -v
bundle -v
rails -v
uname -a

[ksamc]

@stereobooster tried your simplified patch as well but with no luck.

ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

gem -v
2.6.11

bundle -v
Bundler version 1.15.1

rails -v
Rails 5.1.2

uname -a
4.4.0-83-generic #106-Ubuntu

[tagliala]

Same here. Workaround didn't help, downgrading to 3.8.2 works

ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

gem -v
2.6.12

bundle -v
Bundler version 1.15.1

bundle exec rails -v
Rails 5.1.2

uname -a
Linux <NAME> 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[vedant1811]

@grosser https://github.com/stereobooster/ruby-server-experiment/tree/master/puma-symlink-grosser

UPD: @grosser why do you prefer USR2 over USR1 restart?

PS simplified patch

require 'bundler/setup'
on_restart do
  ENV.replace(Bundler.clean_env)
end

@stereobooster the error is different:

As mentioned here, fIx is for:

uninitialized constant #<Class:#<Puma::DSL:0x0055883f37e188>>::Bundler (NameError)

but the error I (and probably others) am getting is:

`find_spec_for_exe': can't find gem puma (>= 0.a) (Gem::GemNotFoundException)

Info:

$ ~/.rvm/bin/rvm ruby-2.4.1 do ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

$ ~/.rvm/bin/rvm ruby-2.4.1 do gem -v
2.6.11

$ ~/.rvm/bin/rvm ruby-2.4.1 do bundle -v
Bundler version 1.15.1

$ ~/.rvm/bin/rvm ruby-2.4.1 do bundle exec rails -v
Rails 5.1.2

$ uname -a                                                                                                                                                                                            
Linux ip-172-31-30-115 4.4.0-83-generic #106-Ubuntu SMP Mon Jun 26 17:54:43 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux 

Okay, using puma-3.8.1 is the clean and simple workaround.
Thanks @dguettler and @tagliala.

[vizcay]

I've just been bitten by this, here is my config:

Ruby: 2.3.3p222
Puma: 3.9.1
Rails: 5.0.1
bundler: 1.15.13
rbenv: 1.1.1-2g615f844
Linux: Ubuntu 16.04

[vizcay]

I can confirm that switching back to puma 3.8.1 solved the issue.

[grosser]

# Gemfile
source 'https://rubygems.org'
gem 'puma', git: 'https://github.com/grosser/puma.git', branch: 'grosser/fix'

# config.ru
run lambda { |env| [200, {'Content-Type'=>'text/plain'}, StringIO.new("Hello World!\n")] }

this setup works fine for me ...
#1385

bundle exec puma
kill -SIGUSR2 <PID>

[dgutov]

The above fix doesn't seem likely to solve the problem reported in the first comment (with dependencies not being found), so forgive me if I'm not testing it.

Instead of downgrading, though, we've settled on the --restart-cmd "bundle exec puma" workaround here. Works quite well.

[grosser]

Yeah, found this bug while trying to reproduce the restarting issue ... mind trying out https://github.com/grosser/puma-restart (using puma from master + my patch) and tell me if that works for you too ?

[dgutov]

@grosser That works. But it works even if I use

gem 'puma', git: 'https://github.com/grosser/puma.git', branch: 'master'

instead of your fix branch.

[dgutov]

And this example fails with 3.9.1 here. Looks like a good news.

[kyontan]

I also reproduced this using with sinatra. (also 3.8.2 works well)

$ ruby -v
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin16]
$ gem -v
2.6.8
$ bundle -v
Bundler version 1.15.3
$ uname -a
Darwin PC103323.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
$ rbenv -v
rbenv 1.1.1-2-g615f844

Using with alexch/rerun

$ rerun --restart --signal SIGUSR2 -- bundle exec puma
...snip...
20:02:23 [rerun] Change detected: 1 modified: app.rb
20:02:24 [rerun] App restarting
20:02:24 [rerun] Sending signal SIGUSR2 to 17364
* Restarting...
/usr/local/Cellar/ruby/2.4.0/lib/ruby/2.4.0/rubygems.rb:270:in `find_spec_for_exe': can't find gem puma (>= 0.a) (Gem::GemNotFoundException)
        from /usr/local/Cellar/ruby/2.4.0/lib/ruby/2.4.0/rubygems.rb:298:in `activate_bin_path'
        from /.../app/vendor/bundle/ruby/2.4.0/bin/puma:22:in `<main>'

[grosser]

can you try with master to see if that fixes it ?

@nateberkopec want to make a release to see if some of these get solved ?

[kyontan]

It seems fixed at a771ec3 !
What's the reason of this issue?

[grosser]

obviously missing Teapot ;) I think it was not passing in -rbundler/setup so the new process tried to load puma without bundler and blew up

…

On Sun, Aug 13, 2017 at 4:36 AM, kyontan ***@***.***> wrote: It seems fixed at a771ec3 <a771ec3> ! What's the reason of this issue? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAsZ6TsSz9wdc5gMEr5eI2eHSxOwsFyks5sXt-2gaJpZM4NtbSQ> .

[nateberkopec]

Closed by #1385