PUMA stops responding after a few minutes idle time. LSOF -wni tcp:port shows CLOSE_WAIT connections

[hamzaaltaf]

Describe the bug
I am trying to setup my VPS on upcloud, I am using ruby 2.4.2. rails 4.2.8 and puma 4.3.5 on ubuntu 18.02. Whenever I run rails s -b -p 3100 or any other port, server runs perfectly fine. But after idle time of max 10-15 mins, it stops responding, doesn't show any error it just stops responding. I have to restart it every time to make it work. I am stuck with this issue for days now. Kindly help me out if anyone has seen this issue before.

Console shows this warning

! WARNING: Detected 1 Thread(s) started in app boot:
[5466] ! #<Thread:0x0000555bcc783f48@/usr/share/rvm/gems/ruby-2.4.2/gems/concurrent-ruby-1.1.6/lib/concurrent-ruby/concurrent/atomic/ruby_thread_local_var.rb:38 sleep_forever> - /usr/share/rvm/gems/ruby-2.4.2/gems/concurrent-ruby-1.1.6/lib/concurrent-ruby/concurrent/atomic/ruby_thread_local_var.rb:40:in pop

When I run lsof -wni tcp:3100 I see a lot of connections with CLOSE_WAIT in front of them.

Here are my PUMA CONFIGURATION.
Puma config:

workers Integer(ENV['WEB_CONCURRENCY'] || 2)
threads_count = Integer(ENV['RAILS_MAX_THREADS'] || 5)
threads threads_count, threads_count

preload_app!

rackup      DefaultRackup
port        ENV['PORT']     || 3000
environment ENV['RACK_ENV'] || 'development'

on_worker_boot do
  # Worker specific setup for Rails 4.1+
  # See: https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#on-worker-boot
  ActiveRecord::Base.establish_connection
end

[nateberkopec]

Without a reproducing application, it won't be possible to assist you. CLOSE_WAIT means the remote has closed the socket, but your application has not, so it's possible (likely even) that the problem lies in your Rack application.

It looks like you're on Linux, so one thing you could try is to upgrade to 5.0.0.beta1 and use pumactl to send a thread-backtraces command and see what lines your workers are stuck on: #2054

[nateberkopec]

Closing for now. I can reopen if you can provide some of the info above ^^^.