Phased restart issue with Ruby default Gem date

[jules-w2]

Hey,

After a bundle update, if some gems have been updated (Rails default gem), the phased restart doesn't work.
With all others gems, all works fine.

On this case, date gem was updated form 3.3.1 to 3.3.2

==> puma_access.log <==
[349982] - Starting phased worker restart, phase: 6
[349982] + Changing to /srv/apps/production/myapp/current
[349982] - Stopping 3373485 for phased upgrade...
[349982] - TERM sent to 3373485...
[243780] + Gemfile in context: /srv/apps/production/myapp/releases/20221213120620/Gemfile
[243780] ! Unable to start worker
[243780] You have already activated date 3.2.2, but your Gemfile requires date 3.3.1. Since date is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports date as a default gem.
[243780] /srv/apps/production/myapp/shared/bundle/ruby/3.1.0/gems/bundler-2.3.22/lib/bundler/runtime.rb:308:in `check_for_activated_spec!

[dentarg]

Are you using extra_runtime_dependencies? If not, does it help? See docs at https://github.com/puma/puma/blob/7b6301009321d527891e4761da0d39a025b9015f/docs/restart.md#additional-notes-1

[jules-w2]

Hey @dentarg,

I dont use any extra_runtime_dependencies and I think I respect all the recommendations for phased restarts because it's works fine with most updated gems... Only specifics gems create this problem (I think these are just with the default Rails gems, like date on my case).

PS : I use capistrano & capistrano-puma to deploy (https://github.com/seuros/capistrano-puma)

#config/puma/production.rb
env      = "production"
app      = "myapp"
app_path = "/srv/apps/#{env}/#{app}"


directory "#{app_path}/current"
bind      "unix://#{app_path}/shared/tmp/sockets/puma.sock"


workers 2

threads 1,10

prune_bundler

#config/deploy.rb
set :puma_systemctl_user,       :system
set :puma_phased_restart,       true
set :puma_service_unit_name,    "puma_#{fetch(:application)}_#{fetch(:stage)}"
set :puma_access_log,           "#{fetch(:deploy_to)}/shared/log/puma_access.log"
set :puma_error_log,            "#{fetch(:deploy_to)}/shared/log/puma_error.log"

# /etc/systemd/system/puma_myapp_production.service



# This file tells systemd how to run Puma as a 24/7 long-running daemon.
#
# Customize this file based on your bundler location, app directory, etc.
# Customize and copy this into /usr/lib/systemd/system (CentOS) or /lib/systemd/system (Ubuntu).
# Then run:
#   - systemctl enable puma_myapp_production
#   - systemctl {start,stop,restart} puma_myapp_production
#
#
# Use `journalctl -u puma_myapp_production -rn 100` to view the last 100 lines of log output.
#
[Unit]
Description=Puma HTTP Server for myapp (production)

After=syslog.target network.target

[Service]
Type=notify
WatchdogSec=10
User=root
WorkingDirectory=/srv/apps/production/myapp/current
ExecStart=/usr/local/rvm/bin/rvm 3.1.2@myapp_production do bundle exec puma -e production
ExecReload=/bin/kill -USR1 $MAINPID

# if we crash, restart
RestartSec=1
Restart=on-failure

StandardOutput=append:/srv/apps/production/myapp/shared/log/puma_access.log
StandardError=append:/srv/apps/production/myapp/shared/log/puma_error.log

SyslogIdentifier=puma_myapp_production
[Install]
WantedBy=multi-user.target

cap production deploy

...
...
01:04 puma:reload
      01 /bin/systemctl status puma_myapp_production > /dev/null
    ✔ 01 root@web01.myserver.com 0.280s
      02 sudo /bin/systemctl reload puma_myapp_production
    ✔ 01 root@web01.myserver.com 0.530s

[dentarg]

(Never mind my earlier comment about extra_runtime_dependencies, but I was too quick thinking it could be used to address this; side-note: we don't have much docs on extra_runtime_dependencies but the PR that contributed it has more on it)

I think this similar to the problems in #2843 and #2893

Puma requires time and that will load the date extension (also seen at #2843 (comment))

puma/lib/puma.rb

Line 6 in 7b63010

require 'time'

arm64 $ ruby -rtime -e '$LOADED_FEATURES.grep(/\.(so|dylib|dll|bundle)\z/i).each { |f| puts("    #{f}") }'
    enumerator.so
    fiber.so
    rational.so
    complex.so
    /Users/dentarg/.arm64_rubies/3.1.2/lib/ruby/3.1.0/arm64-darwin21/enc/encdb.bundle
    /Users/dentarg/.arm64_rubies/3.1.2/lib/ruby/3.1.0/arm64-darwin21/enc/trans/transdb.bundle
    /Users/dentarg/.arm64_rubies/3.1.2/lib/ruby/3.1.0/arm64-darwin21/monitor.bundle
    /Users/dentarg/.arm64_rubies/3.1.2/lib/ruby/gems/3.1.0/gems/date-3.3.1/lib/date_core.bundle

According to the phased restart docs

Phased restarts cannot be used to upgrade any gems loaded by the Puma master process

I think you are running into that limitation of phased restart.

I just did rails new on my own computer and it looks like the dependancy on date comes from net-imap (which comes from the mail that actionmailbox depend on).

mail 2.7.1 did not depend on net-imap so I guess that's why you are seeing this now.

[dentarg]

This prompted me to create a poll in the discussion forum: #3034

[MSP-Greg]

@jules-w2

About eleven years ago, requiring time and etc was added to Puma. time is still used today, but only a small part of it. It also loads date, which loads date_core.so. PR #3035 removes the need for both time and etc. When it hits a release, your problem should be fixed.

In the meantime, if you can lock to the version of date included in the Ruby you're using, that might cure the problem...