diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index ca6f06dc2a..4effb5a4f1 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -5,7 +5,7 @@ Please describe your pull request. Thank you for contributing! You're the best. - [ ] I have reviewed the [guidelines for contributing](../blob/master/CONTRIBUTING.md) to this repository. -- [ ] I have added an entry to [History.md](../blob/master/History.md) if this PR fixes a bug or adds a feature. If it doesn't need an entry to HISTORY.md, I have added `[changelog skip]` the pull request title. +- [ ] I have added an entry to [History.md](../blob/master/History.md) if this PR fixes a bug or adds a feature. If it doesn't need an entry to HISTORY.md, I have added `[changelog skip]` or `[ci skip]` to the pull request title. - [ ] I have added appropriate tests if this PR fixes a bug or adds a feature. - [ ] My pull request is 100 lines added/removed or less so that it can be easily reviewed. - [ ] If this PR doesn't need tests (docs change), I added `[ci skip]` to the title of the PR. diff --git a/.github/workflows/puma.yml b/.github/workflows/puma.yml index fe5132c886..a96e801b9c 100644 --- a/.github/workflows/puma.yml +++ b/.github/workflows/puma.yml @@ -37,7 +37,6 @@ jobs: uses: MSP-Greg/setup-ruby-pkgs@v1 with: ruby-version: ${{ matrix.ruby }} - bundler: 1 apt-get: ragel brew: ragel mingw: _upgrade_ openssl ragel @@ -49,13 +48,12 @@ jobs: if ('${{ matrix.ruby }}' -lt '2.3') { gem update --system 2.7.10 --no-document } - bundle install --jobs 4 --retry 3 --path=.bundle/puma + bundle install --jobs 4 --retry 3 - name: compile run: bundle exec rake compile - name: rubocop - if: startsWith(matrix.ruby, '2.') run: bundle exec rake rubocop - name: test diff --git a/Gemfile b/Gemfile index 053d2cb0ed..521d91d832 100644 --- a/Gemfile +++ b/Gemfile @@ -3,7 +3,7 @@ source "https://rubygems.org" gemspec gem "rdoc" -gem "rake-compiler" +gem "rake-compiler", "~> 0.9.4" gem "nio4r", "~> 2.0" gem "rack", "~> 1.6" diff --git a/History.md b/History.md index a6d4745468..fbb1ffe73b 100644 --- a/History.md +++ b/History.md @@ -1,3 +1,9 @@ +### Master +* Bugfixes + * Resolve issue with threadpool waiting counter decrement when thread is killed + * Constrain rake-compiler version to 0.9.4 to fix `ClassNotFound` exception when using MiniSSL with Java8. + * Ensure that TCP_CORK is usable + ## 5.0.0 * Features @@ -6,11 +12,12 @@ * EXPERIMENTAL: Added `nakayoshi_fork` option. Reduce memory usage in preloaded cluster-mode apps by GCing before fork and compacting, where available. (#2093, #2256) * Added pumactl `thread-backtraces` command to print thread backtraces (#2054) * Added incrementing `requests_count` to `Puma.stats`. (#2106) - * Increased maximum URI path length from 2048 to 8196 bytes (#2167) + * Increased maximum URI path length from 2048 to 8192 bytes (#2167, #2344) * `lowlevel_error_handler` is now called during a forced threadpool shutdown, and if a callable with 3 arguments is set, we now also pass the status code (#2203) * Faster phased restart and worker timeout (#2220) * Added `state_permission` to config DSL to set state file permissions (#2238) * Added `Puma.stats_hash`, which returns a stats in Hash instead of a JSON string (#2086, #2253) + * `rack.multithread` and `rack.multiprocess` now dynamically resolved by `max_thread` and `workers` respectively (#2288) * Deprecations, Removals and Breaking API Changes * `--control` has been removed. Use `--control-url` (#1487) @@ -24,9 +31,11 @@ * Daemonization has been removed without replacement. (#2170) * Changed #connected_port to #connected_ports (#2076) * Configuration: `environment` is read from `RAILS_ENV`, if `RACK_ENV` can't be found (#2022) + * Log binding on http:// for TCP bindings to make it clickable * Bugfixes * Fix JSON loading issues on phased-restarts (#2269) + * Improve shutdown reliability (#2312, #2338) * Close client http connections made to an ssl server with TLSv1.3 (#2116) * Do not set user_config to quiet by default to allow for file config (#2074) * Always close SSL connection in Puma::ControlCLI (#2211) @@ -46,6 +55,8 @@ * Fix `UserFileDefaultOptions#fetch` to properly use `default` (#2233) * Improvements to `out_of_band` hook (#2234) * Prefer the rackup file specified by the CLI (#2225) + * Fix for spawning subprocesses with fork_worker option (#2267) + * Set `CONTENT_LENGTH` for chunked requests (#2287) * Refactor * Remove unused loader argument from Plugin initializer (#2095) @@ -58,6 +69,13 @@ * Support parallel tests in verbose progress reporting (#2223) * Refactor error handling in server accept loop (#2239) +## 4.3.4/4.3.5 and 3.12.5/3.12.6 / 2020-05-22 + +Each patchlevel release contains a separate security fix. We recommend simply upgrading to 4.3.5/3.12.6. + +* Security + * Fix: Fixed two separate HTTP smuggling vulnerabilities that used the Transfer-Encoding header. CVE-2020-11076 and CVE-2020-11077. + ## 4.3.3 and 3.12.4 / 2020-02-28 * Bugfixes diff --git a/README.md b/README.md index acdbdc143c..63bf299e8d 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ [![Code Climate](https://codeclimate.com/github/puma/puma.svg)](https://codeclimate.com/github/puma/puma) [![SemVer](https://api.dependabot.com/badges/compatibility_score?dependency-name=puma&package-manager=bundler&version-scheme=semver)](https://dependabot.com/compatibility-score.html?dependency-name=puma&package-manager=bundler&version-scheme=semver) -[![StackOverflow](http://img.shields.io/badge/stackoverflow-Puma-blue.svg)]( http://stackoverflow.com/questions/tagged/puma ) +[![StackOverflow](https://img.shields.io/badge/stackoverflow-Puma-blue.svg)]( https://stackoverflow.com/questions/tagged/puma ) Puma is a **simple, fast, multi-threaded, and highly concurrent HTTP 1.1 server for Ruby/Rack applications**. @@ -27,7 +27,7 @@ $ gem install puma $ puma ``` -Without arguments, puma will look for a rackup (.ru) file in +Without arguments, puma will look for a rackup (.ru) file in working directory called `config.ru`. ## Frameworks @@ -135,7 +135,7 @@ Preloading can’t be used with phased restart, since phased restart kills and r If puma encounters an error outside of the context of your application, it will respond with a 500 and a simple textual error message (see `lowlevel_error` in [this file](https://github.com/puma/puma/blob/master/lib/puma/server.rb)). You can specify custom behavior for this scenario. For example, you can report the error to your third-party -error-tracking service (in this example, [rollbar](http://rollbar.com)): +error-tracking service (in this example, [rollbar](https://rollbar.com)): ```ruby lowlevel_error_handler do |e| diff --git a/docs/architecture.md b/docs/architecture.md index 958c6a25ea..b5d6451f7c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,7 +2,7 @@ ## Overview -![http://bit.ly/2iJuFky](images/puma-general-arch.png) +![https://bit.ly/2iJuFky](images/puma-general-arch.png) Puma is a threaded web server, processing requests across a TCP or UNIX socket. @@ -12,7 +12,7 @@ Clustered mode is shown/discussed here. Single mode is analogous to having a sin ## Connection pipeline -![http://bit.ly/2zwzhEK](images/puma-connection-flow.png) +![https://bit.ly/2zwzhEK](images/puma-connection-flow.png) * Upon startup, Puma listens on a TCP or UNIX socket. * The backlog of this socket is configured (with a default of 1024), determining how many established but unaccepted connections can exist concurrently. @@ -29,7 +29,7 @@ Clustered mode is shown/discussed here. Single mode is analogous to having a sin ### Disabling `queue_requests` -![http://bit.ly/2zxCJ1Z](images/puma-connection-flow-no-reactor.png) +![https://bit.ly/2zxCJ1Z](images/puma-connection-flow-no-reactor.png) The `queue_requests` option is `true` by default, enabling the separate thread used to buffer requests as described above. diff --git a/docs/deployment.md b/docs/deployment.md index c237f93080..29d1b4a281 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -20,7 +20,10 @@ Welcome back! Puma was originally conceived as a thread-only webserver, but grew the ability to also use processes in version 2. -Here are some rules of thumb: +To run puma in single mode (e.g. for a development environment) you will need to +set the number of workers to 0, anything above will run in cluster mode. + +Here are some rules of thumb for cluster mode: ### MRI @@ -66,7 +69,8 @@ thread to become available. * Have your upstream proxy set a header with the time it received the request: * nginx: `proxy_set_header X-Request-Start "${msec}";` - * haproxy: `http-request set-header X-Request-Start "%t";` + * haproxy >= 1.9: `http-request set-header X-Request-Start t=%[date()]%[date_us()]` + * haproxy < 1.9: `http-request set-header X-Request-Start t=%[date()]` * In your Rack middleware, determine the amount of time elapsed since `X-Request-Start`. * To improve accuracy, you will want to subtract time spent waiting for slow clients: * `env['puma.request_body_wait']` contains the number of milliseconds Puma spent diff --git a/docs/signals.md b/docs/signals.md index 9661aa5956..7026927bb4 100644 --- a/docs/signals.md +++ b/docs/signals.md @@ -1,8 +1,8 @@ -The [unix signal](http://en.wikipedia.org/wiki/Unix_signal) is a method of sending messages between [processes](http://en.wikipedia.org/wiki/Process_(computing)). When a signal is sent, the operating system interrupts the target process's normal flow of execution. There are standard signals that are used to stop a process but there are also custom signals that can be used for other purposes. This document is an attempt to list all supported signals that Puma will respond to. In general, signals need only be sent to the master process of a cluster. +The [unix signal](https://en.wikipedia.org/wiki/Unix_signal) is a method of sending messages between [processes](https://en.wikipedia.org/wiki/Process_(computing)). When a signal is sent, the operating system interrupts the target process's normal flow of execution. There are standard signals that are used to stop a process but there are also custom signals that can be used for other purposes. This document is an attempt to list all supported signals that Puma will respond to. In general, signals need only be sent to the master process of a cluster. ## Sending Signals -If you are new to signals it can be useful to see how they can be used. When a process is created in a *nix like operating system it will have a [PID - or process identifier](http://en.wikipedia.org/wiki/Process_identifier) that can be used to send signals to the process. For demonstration we will create an infinitely running process by tailing a file: +If you are new to signals it can be useful to see how they can be used. When a process is created in a *nix like operating system it will have a [PID - or process identifier](https://en.wikipedia.org/wiki/Process_identifier) that can be used to send signals to the process. For demonstration we will create an infinitely running process by tailing a file: ```sh $ echo "foo" >> my.log @@ -17,13 +17,13 @@ $ ps aux | grep tail schneems 87152 0.0 0.0 2432772 492 s032 S+ 12:46PM 0:00.00 tail -f my.log ``` -You can send a signal in Ruby using the [Process module](http://www.ruby-doc.org/core-2.1.1/Process.html#kill-method): +You can send a signal in Ruby using the [Process module](https://www.ruby-doc.org/core-2.1.1/Process.html#kill-method): ``` $ irb > puts pid => 87152 -Process.detach(pid) # http://ruby-doc.org/core-2.1.1/Process.html#method-c-detach +Process.detach(pid) # https://ruby-doc.org/core-2.1.1/Process.html#method-c-detach Process.kill("TERM", pid) ``` diff --git a/ext/puma_http11/http11_parser.c b/ext/puma_http11/http11_parser.c index 0b5fdabc3b..bf1dd89ab9 100644 --- a/ext/puma_http11/http11_parser.c +++ b/ext/puma_http11/http11_parser.c @@ -14,12 +14,14 @@ /* * capitalizes all lower-case ASCII characters, - * converts dashes to underscores. + * converts dashes to underscores, and underscores to commas. */ static void snake_upcase_char(char *c) { if (*c >= 'a' && *c <= 'z') *c &= ~0x20; + else if (*c == '_') + *c = ','; else if (*c == '-') *c = '_'; } diff --git a/ext/puma_http11/http11_parser.rl b/ext/puma_http11/http11_parser.rl index 880c1d40be..62452ba7c2 100644 --- a/ext/puma_http11/http11_parser.rl +++ b/ext/puma_http11/http11_parser.rl @@ -12,12 +12,14 @@ /* * capitalizes all lower-case ASCII characters, - * converts dashes to underscores. + * converts dashes to underscores, and underscores to commas. */ static void snake_upcase_char(char *c) { if (*c >= 'a' && *c <= 'z') *c &= ~0x20; + else if (*c == '_') + *c = ','; else if (*c == '-') *c = '_'; } diff --git a/ext/puma_http11/org/jruby/puma/MiniSSL.java b/ext/puma_http11/org/jruby/puma/MiniSSL.java index c8b929658c..33b6d91a4c 100644 --- a/ext/puma_http11/org/jruby/puma/MiniSSL.java +++ b/ext/puma_http11/org/jruby/puma/MiniSSL.java @@ -173,7 +173,7 @@ public IRubyObject initialize(ThreadContext threadContext, IRubyObject miniSSLCo engine.setEnabledProtocols(protocols); engine.setUseClientMode(false); - long verify_mode = miniSSLContext.callMethod(threadContext, "verify_mode").convertToInteger().getLongValue(); + long verify_mode = miniSSLContext.callMethod(threadContext, "verify_mode").convertToInteger("to_i").getLongValue(); if ((verify_mode & 0x1) != 0) { // 'peer' engine.setWantClientAuth(true); } diff --git a/ext/puma_http11/puma_http11.c b/ext/puma_http11/puma_http11.c index a2aedda642..962cb8476e 100644 --- a/ext/puma_http11/puma_http11.c +++ b/ext/puma_http11/puma_http11.c @@ -54,7 +54,7 @@ DEF_MAX_LENGTH(FIELD_NAME, 256); DEF_MAX_LENGTH(FIELD_VALUE, 80 * 1024); DEF_MAX_LENGTH(REQUEST_URI, 1024 * 12); DEF_MAX_LENGTH(FRAGMENT, 1024); /* Don't know if this length is specified somewhere or not */ -DEF_MAX_LENGTH(REQUEST_PATH, 8196); +DEF_MAX_LENGTH(REQUEST_PATH, 8192); DEF_MAX_LENGTH(QUERY_STRING, (1024 * 10)); DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32))); diff --git a/lib/puma/binder.rb b/lib/puma/binder.rb index 7759c85b66..ef9c24fb1a 100644 --- a/lib/puma/binder.rb +++ b/lib/puma/binder.rb @@ -6,6 +6,7 @@ require 'puma/const' require 'puma/util' require 'puma/minissl/context_builder' +require 'puma/configuration' module Puma class Binder @@ -13,7 +14,7 @@ class Binder RACK_VERSION = [1,6].freeze - def initialize(events) + def initialize(events, conf = Configuration.new) @events = events @listeners = [] @inherited_fds = {} @@ -23,8 +24,8 @@ def initialize(events) @proto_env = { "rack.version".freeze => RACK_VERSION, "rack.errors".freeze => events.stderr, - "rack.multithread".freeze => true, - "rack.multiprocess".freeze => false, + "rack.multithread".freeze => conf.options[:max_threads] > 1, + "rack.multiprocess".freeze => conf.options[:workers] >= 1, "rack.run_once".freeze => false, "SCRIPT_NAME".freeze => ENV['SCRIPT_NAME'] || "", @@ -113,7 +114,7 @@ def parse(binds, logger, log_msg = 'Listening') i.local_address.ip_unpack.join(':') end - logger.log "* #{log_msg} on tcp://#{addr}" + logger.log "* #{log_msg} on http://#{addr}" end end diff --git a/lib/puma/client.rb b/lib/puma/client.rb index 324947b4b0..f49bba6176 100644 --- a/lib/puma/client.rb +++ b/lib/puma/client.rb @@ -308,8 +308,16 @@ def setup_body te = @env[TRANSFER_ENCODING2] - if te && CHUNKED.casecmp(te) == 0 - return setup_chunked_body(body) + if te + if te.include?(",") + te.split(",").each do |part| + if CHUNKED.casecmp(part.strip) == 0 + return setup_chunked_body(body) + end + end + elsif CHUNKED.casecmp(te) == 0 + return setup_chunked_body(body) + end end @chunked_body = false @@ -412,7 +420,10 @@ def read_chunked_body raise EOFError end - return true if decode_chunk(chunk) + if decode_chunk(chunk) + @env[CONTENT_LENGTH] = @chunked_content_length + return true + end end end @@ -424,20 +435,28 @@ def setup_chunked_body(body) @body = Tempfile.new(Const::PUMA_TMP_BASE) @body.binmode @tempfile = @body + @chunked_content_length = 0 + + if decode_chunk(body) + @env[CONTENT_LENGTH] = @chunked_content_length + return true + end + end - return decode_chunk(body) + def write_chunk(str) + @chunked_content_length += @body.write(str) end def decode_chunk(chunk) if @partial_part_left > 0 if @partial_part_left <= chunk.size if @partial_part_left > 2 - @body << chunk[0..(@partial_part_left-3)] # skip the \r\n + write_chunk(chunk[0..(@partial_part_left-3)]) # skip the \r\n end chunk = chunk[@partial_part_left..-1] @partial_part_left = 0 else - @body << chunk if @partial_part_left > 2 # don't include the last \r\n + write_chunk(chunk) if @partial_part_left > 2 # don't include the last \r\n @partial_part_left -= chunk.size return false end @@ -484,12 +503,12 @@ def decode_chunk(chunk) case when got == len - @body << part[0..-3] # to skip the ending \r\n + write_chunk(part[0..-3]) # to skip the ending \r\n when got <= len - 2 - @body << part + write_chunk(part) @partial_part_left = len - part.size when got == len - 1 # edge where we get just \r but not \n - @body << part[0..-2] + write_chunk(part[0..-2]) @partial_part_left = len - part.size end else diff --git a/lib/puma/cluster.rb b/lib/puma/cluster.rb index aff32ab885..7e9dd95b20 100644 --- a/lib/puma/cluster.rb +++ b/lib/puma/cluster.rb @@ -248,6 +248,7 @@ def worker(index, master) $0 = title Signal.trap "SIGINT", "IGNORE" + Signal.trap "SIGCHLD", "DEFAULT" fork_worker = @options[:fork_worker] && index == 0 @@ -284,9 +285,11 @@ def worker(index, master) if fork_worker restart_server.clear + worker_pids = [] Signal.trap "SIGCHLD" do - Process.wait(-1, Process::WNOHANG) rescue nil - wakeup! + wakeup! if worker_pids.reject! do |p| + Process.wait(p, Process::WNOHANG) rescue true + end end Thread.new do @@ -303,7 +306,7 @@ def worker(index, master) elsif idx == 0 # restart server restart_server << true << false else # fork worker - pid = spawn_worker(idx, master) + worker_pids << pid = spawn_worker(idx, master) @worker_write << "f#{pid}:#{idx}\n" rescue nil end end diff --git a/lib/puma/commonlogger.rb b/lib/puma/commonlogger.rb index 25989e7245..4762be30dc 100644 --- a/lib/puma/commonlogger.rb +++ b/lib/puma/commonlogger.rb @@ -3,7 +3,7 @@ module Puma # Rack::CommonLogger forwards every request to the given +app+, and # logs a line in the - # {Apache common log format}[http://httpd.apache.org/docs/1.3/logs.html#common] + # {Apache common log format}[https://httpd.apache.org/docs/1.3/logs.html#common] # to the +logger+. # # If +logger+ is nil, CommonLogger will fall back +rack.errors+, which is @@ -16,7 +16,7 @@ module Puma # (which is called without arguments in order to make the error appear for # sure) class CommonLogger - # Common Log Format: http://httpd.apache.org/docs/1.3/logs.html#common + # Common Log Format: https://httpd.apache.org/docs/1.3/logs.html#common # # lilith.local - - [07/Aug/2006 23:58:02 -0400] "GET / HTTP/1.1" 500 - # diff --git a/lib/puma/dsl.rb b/lib/puma/dsl.rb index c11b130cdc..4dbe4c3e75 100644 --- a/lib/puma/dsl.rb +++ b/lib/puma/dsl.rb @@ -443,8 +443,8 @@ def before_fork(&block) # # @note Cluster mode only. # @example - # on_worker_fork do - # puts 'Before worker fork...' + # on_worker_boot do + # puts 'Before worker boot...' # end def on_worker_boot(&block) @options[:before_worker_boot] ||= [] @@ -769,7 +769,7 @@ def fork_worker(after_requests=1000) # also increase time to boot and fork. See your logs for details on how much # time this adds to your boot process. For most apps, it will be less than one # second. - def nakayoshi_fork(enabled=false) + def nakayoshi_fork(enabled=true) @options[:nakayoshi_fork] = enabled end end diff --git a/lib/puma/error_logger.rb b/lib/puma/error_logger.rb new file mode 100644 index 0000000000..e9f2ad6478 --- /dev/null +++ b/lib/puma/error_logger.rb @@ -0,0 +1,96 @@ +# frozen_string_literal: true + +require 'puma/const' + +module Puma + # The implementation of a detailed error logging. + # + class ErrorLogger + include Const + + attr_reader :ioerr + + REQUEST_FORMAT = %{"%s %s%s" - (%s)} + + def initialize(ioerr) + @ioerr = ioerr + @ioerr.sync = true + + @debug = ENV.key? 'PUMA_DEBUG' + end + + def self.stdio + new $stderr + end + + # Print occured error details. + # +options+ hash with additional options: + # - +error+ is an exception object + # - +req+ the http request + # - +text+ (default nil) custom string to print in title + # and before all remaining info. + # + def info(options={}) + ioerr.puts title(options) + end + + # Print occured error details only if + # environment variable PUMA_DEBUG is defined. + # +options+ hash with additional options: + # - +error+ is an exception object + # - +req+ the http request + # - +text+ (default nil) custom string to print in title + # and before all remaining info. + # + def debug(options={}) + return unless @debug + + error = options[:error] + req = options[:req] + + string_block = [] + string_block << title(options) + string_block << request_dump(req) if req + string_block << error_backtrace(options) if error + + ioerr.puts string_block.join("\n") + end + + def title(options={}) + text = options[:text] + req = options[:req] + error = options[:error] + + string_block = ["#{Time.now}"] + string_block << " #{text}" if text + string_block << " (#{request_title(req)})" if request_parsed?(req) + string_block << ": #{error.inspect}" if error + string_block.join('') + end + + def request_dump(req) + "Headers: #{request_headers(req)}\n" \ + "Body: #{req.body}" + end + + def request_title(req) + env = req.env + + REQUEST_FORMAT % [ + env[REQUEST_METHOD], + env[REQUEST_PATH] || env[PATH_INFO], + env[QUERY_STRING] || "", + env[HTTP_X_FORWARDED_FOR] || env[REMOTE_ADDR] || "-" + ] + end + + def request_headers(req) + headers = req.env.select { |key, _| key.start_with?('HTTP_') } + headers.map { |key, value| [key[5..-1], value] }.to_h.inspect + end + + def request_parsed?(req) + req && req.env[REQUEST_METHOD] + end + end +end diff --git a/lib/puma/events.rb b/lib/puma/events.rb index fac0c9f879..c746c9d1a5 100644 --- a/lib/puma/events.rb +++ b/lib/puma/events.rb @@ -1,7 +1,7 @@ # frozen_string_literal: true -require 'puma/const' require "puma/null_io" +require 'puma/error_logger' require 'stringio' module Puma @@ -23,8 +23,6 @@ def call(str) end end - include Const - # Create an Events object that prints to +stdout+ and +stderr+. # def initialize(stdout, stderr) @@ -36,6 +34,7 @@ def initialize(stdout, stderr) @stderr.sync = true @debug = ENV.key? 'PUMA_DEBUG' + @error_logger = ErrorLogger.new(@stderr) @hooks = Hash.new { |h,k| h[k] = [] } end @@ -66,7 +65,8 @@ def register(hook, obj=nil, &blk) # Write +str+ to +@stdout+ # def log(str) - @stdout.puts format(str) + @stdout.puts format(str) if @stdout.respond_to? :puts + rescue Errno::EPIPE end def write(str) @@ -80,7 +80,7 @@ def debug(str) # Write +str+ to +@stderr+ # def error(str) - @stderr.puts format("ERROR: #{str}") + @error_logger.info(text: format("ERROR: #{str}")) exit 1 end @@ -88,42 +88,45 @@ def format(str) formatter.call(str) end + # An HTTP connection error has occurred. + # +error+ a connection exception, +req+ the request, + # and +text+ additional info + # + def connection_error(error, req, text="HTTP connection error") + @error_logger.info(error: error, req: req, text: text) + end + # An HTTP parse error has occurred. - # +server+ is the Server object, +env+ the request, and +error+ a - # parsing exception. + # +error+ a parsing exception, + # and +req+ the request. # - def parse_error(server, env, error) - @stderr.puts "#{Time.now}: HTTP parse error, malformed request " \ - "(#{env[HTTP_X_FORWARDED_FOR] || env[REMOTE_ADDR]}#{env[REQUEST_PATH]}): " \ - "#{error.inspect}" + def parse_error(error, req) + @error_logger.info(error: error, req: req, text: 'HTTP parse error, malformed request') end # An SSL error has occurred. - # +server+ is the Server object, +peeraddr+ peer address, +peercert+ - # any peer certificate (if present), and +error+ an exception object. + # +error+ an exception object, +peeraddr+ peer address, + # and +peercert+ any peer certificate (if present). # - def ssl_error(server, peeraddr, peercert, error) + def ssl_error(error, peeraddr, peercert) subject = peercert ? peercert.subject : nil - @stderr.puts "#{Time.now}: SSL error, peer: #{peeraddr}, peer cert: #{subject}, #{error.inspect}" + @error_logger.info(error: error, text: "SSL error, peer: #{peeraddr}, peer cert: #{subject}") end # An unknown error has occurred. - # +server+ is the Server object, +error+ an exception object, - # +kind+ some additional info, and +env+ the request. + # +error+ an exception object, +req+ the request, + # and +text+ additional info # - def unknown_error(server, error, kind="Unknown", env=nil) - if error.respond_to? :render - error.render "#{Time.now}: #{kind} error", @stderr - else - if env - string_block = [ "#{Time.now}: #{kind} error handling request { #{env['REQUEST_METHOD']} #{env['PATH_INFO']} }" ] - string_block << error.inspect - else - string_block = [ "#{Time.now}: #{kind} error: #{error.inspect}" ] - end - string_block << error.backtrace - @stderr.puts string_block.join("\n") - end + def unknown_error(error, req=nil, text="Unknown error") + @error_logger.info(error: error, req: req, text: text) + end + + # Log occurred error debug dump. + # +error+ an exception object, +req+ the request, + # and +text+ additional info + # + def debug_error(error, req=nil, text="") + @error_logger.debug(error: error, req: req, text: text) end def on_booted(&block) diff --git a/lib/puma/launcher.rb b/lib/puma/launcher.rb index 0c4e1dd304..3ec065aa5e 100644 --- a/lib/puma/launcher.rb +++ b/lib/puma/launcher.rb @@ -47,7 +47,7 @@ def initialize(conf, launcher_args={}) @original_argv = @argv.dup @config = conf - @binder = Binder.new(@events) + @binder = Binder.new(@events, conf) @binder.create_inherited_fds(ENV).each { |k| ENV.delete k } @binder.create_activated_fds(ENV).each { |k| ENV.delete k } @@ -111,6 +111,7 @@ def write_state sf.pid = Process.pid sf.control_url = @options[:control_url] sf.control_auth_token = @options[:control_auth_token] + sf.running_from = File.expand_path('.') sf.save path, permission end @@ -172,12 +173,13 @@ def run case @status when :halt log "* Stopping immediately!" + @runner.stop_control when :run, :stop graceful_stop when :restart log "* Restarting..." ENV.replace(previous_env) - @runner.before_restart + @runner.stop_control restart! when :exit # nothing diff --git a/lib/puma/reactor.rb b/lib/puma/reactor.rb index 7a4fce5cad..780903d296 100644 --- a/lib/puma/reactor.rb +++ b/lib/puma/reactor.rb @@ -252,7 +252,7 @@ def run_internal c.close clear_monitor mon - @events.ssl_error @server, addr, cert, e + @events.ssl_error e, addr, cert # The client doesn't know HTTP well rescue HttpParserError => e @@ -263,7 +263,7 @@ def run_internal clear_monitor mon - @events.parse_error @server, c.env, e + @events.parse_error e, c rescue StandardError => e @server.lowlevel_error(e, c.env) diff --git a/lib/puma/runner.rb b/lib/puma/runner.rb index 4695aa9803..4a9a9c2ae4 100644 --- a/lib/puma/runner.rb +++ b/lib/puma/runner.rb @@ -30,7 +30,7 @@ def log(str) @events.log str end - def before_restart + def stop_control @control.stop(true) if @control end diff --git a/lib/puma/server.rb b/lib/puma/server.rb index 70095577eb..b9fa8f6851 100644 --- a/lib/puma/server.rb +++ b/lib/puma/server.rb @@ -98,10 +98,22 @@ def inherit_binder(bind) @binder = bind end + class << self + # :nodoc: + def tcp_cork_supported? + RbConfig::CONFIG['host_os'] =~ /linux/ && + Socket.const_defined?(:IPPROTO_TCP) && + Socket.const_defined?(:TCP_CORK) && + Socket.const_defined?(:SOL_TCP) && + Socket.const_defined?(:TCP_INFO) + end + private :tcp_cork_supported? + end + # On Linux, use TCP_CORK to better control how the TCP stack # packetizes our stream. This improves both latency and throughput. # - if RUBY_PLATFORM =~ /linux/ + if tcp_cork_supported? UNPACK_TCP_STATE_FROM_TCP_INFO = "C".freeze # 6 == Socket::IPPROTO_TCP @@ -109,7 +121,7 @@ def inherit_binder(bind) # 1/0 == turn on/off def cork_socket(socket) begin - socket.setsockopt(6, 3, 1) if socket.kind_of? TCPSocket + socket.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_CORK, 1) if socket.kind_of? TCPSocket rescue IOError, SystemCallError Thread.current.purge_interrupt_queue if Thread.current.respond_to? :purge_interrupt_queue end @@ -117,7 +129,7 @@ def cork_socket(socket) def uncork_socket(socket) begin - socket.setsockopt(6, 3, 0) if socket.kind_of? TCPSocket + socket.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_CORK, 0) if socket.kind_of? TCPSocket rescue IOError, SystemCallError Thread.current.purge_interrupt_queue if Thread.current.respond_to? :purge_interrupt_queue end @@ -207,14 +219,16 @@ def run(background=true) client.close - @events.ssl_error self, addr, cert, e + @events.ssl_error e, addr, cert rescue HttpParserError => e client.write_error(400) client.close - @events.parse_error self, client.env, e - rescue ConnectionError, EOFError + @events.parse_error e, client + rescue ConnectionError, EOFError => e client.close + + @events.connection_error e, client else if process_now process_client client, buffer @@ -300,7 +314,7 @@ def handle_servers end end rescue Object => e - @events.unknown_error self, e, "Listen loop" + @events.unknown_error e, nil, "Listen loop" end end @@ -313,10 +327,14 @@ def handle_servers end graceful_shutdown if @status == :stop || @status == :restart rescue Exception => e - STDERR.puts "Exception handling servers: #{e.message} (#{e.class})" - STDERR.puts e.backtrace + @events.unknown_error e, nil, "Exception handling servers" ensure - @check.close unless @check.closed? # Ruby 2.2 issue + begin + @check.close unless @check.closed? + rescue Errno::EBADF, RuntimeError + # RuntimeError is Ruby 2.2 issue, can't modify frozen IOError + # Errno::EBADF is infrequently raised + end @notify.close @notify = nil @check = nil @@ -406,7 +424,7 @@ def process_client(client, buffer) close_socket = true - @events.ssl_error self, addr, cert, e + @events.ssl_error e, addr, cert # The client doesn't know HTTP well rescue HttpParserError => e @@ -414,7 +432,7 @@ def process_client(client, buffer) client.write_error(400) - @events.parse_error self, client.env, e + @events.parse_error e, client # Server error rescue StandardError => e @@ -422,8 +440,7 @@ def process_client(client, buffer) client.write_error(500) - @events.unknown_error self, e, "Read" - + @events.unknown_error e, nil, "Read" ensure buffer.reset @@ -433,7 +450,7 @@ def process_client(client, buffer) Thread.current.purge_interrupt_queue if Thread.current.respond_to? :purge_interrupt_queue # Already closed rescue StandardError => e - @events.unknown_error self, e, "Client" + @events.unknown_error e, nil, "Client" end end end @@ -469,7 +486,7 @@ def normalize_env(env, client) env[PATH_INFO] = env[REQUEST_PATH] - # From http://www.ietf.org/rfc/rfc3875 : + # From https://www.ietf.org/rfc/rfc3875 : # "Script authors should be aware that the REMOTE_ADDR and # REMOTE_HOST meta-variables (see sections 4.1.8 and 4.1.9) # may not identify the ultimate source of the request. @@ -558,12 +575,44 @@ def handle_request(req, lines) end fast_write client, "\r\n".freeze - rescue ConnectionError + rescue ConnectionError => e + @events.debug_error e # noop, if we lost the socket we just won't send the early hints end } end + # Fixup any headers with , in the name to have _ now. We emit + # headers with , in them during the parse phase to avoid ambiguity + # with the - to _ conversion for critical headers. But here for + # compatibility, we'll convert them back. This code is written to + # avoid allocation in the common case (ie there are no headers + # with , in their names), that's why it has the extra conditionals. + + to_delete = nil + to_add = nil + + env.each do |k,v| + if k.start_with?("HTTP_") and k.include?(",") and k != "HTTP_TRANSFER,ENCODING" + if to_delete + to_delete << k + else + to_delete = [k] + end + + unless to_add + to_add = {} + end + + to_add[k.tr(",", "_")] = v + end + end + + if to_delete + to_delete.each { |k| env.delete(k) } + env.merge! to_add + end + # A rack extension. If the app writes #call'ables to this # array, we will invoke them when the request is done. # @@ -585,12 +634,12 @@ def handle_request(req, lines) return :async end rescue ThreadPool::ForceShutdown => e - @events.unknown_error self, e, "Rack app", env + @events.unknown_error e, req, "Rack app" @events.log "Detected force shutdown of a thread" status, headers, res_body = lowlevel_error(e, env, 503) rescue Exception => e - @events.unknown_error self, e, "Rack app", env + @events.unknown_error e, req, "Rack app" status, headers, res_body = lowlevel_error(e, env, 500) end @@ -880,7 +929,7 @@ def notify_safely(message) @check, @notify = Puma::Util.pipe unless @notify begin @notify << message - rescue IOError + rescue IOError, NoMethodError, Errno::EPIPE # The server, in another thread, is shutting down Thread.current.purge_interrupt_queue if Thread.current.respond_to? :purge_interrupt_queue rescue RuntimeError => e diff --git a/lib/puma/state_file.rb b/lib/puma/state_file.rb index 6aa8fad4d6..9ea74a8ca8 100644 --- a/lib/puma/state_file.rb +++ b/lib/puma/state_file.rb @@ -19,7 +19,7 @@ def load(path) @options = YAML.load File.read(path) end - FIELDS = %w!control_url control_auth_token pid! + FIELDS = %w!control_url control_auth_token pid running_from! FIELDS.each do |f| define_method f do diff --git a/lib/puma/thread_pool.rb b/lib/puma/thread_pool.rb index c0bbedb1c8..bb6a73c73a 100644 --- a/lib/puma/thread_pool.rb +++ b/lib/puma/thread_pool.rb @@ -122,8 +122,11 @@ def spawn_thread @out_of_band_pending = false end not_full.signal - not_empty.wait mutex - @waiting -= 1 + begin + not_empty.wait mutex + ensure + @waiting -= 1 + end end work = todo.shift diff --git a/puma.gemspec b/puma.gemspec index fad0d34c79..44d16608b9 100644 --- a/puma.gemspec +++ b/puma.gemspec @@ -15,13 +15,13 @@ Gem::Specification.new do |s| end s.files = `git ls-files -- bin docs ext lib tools`.split("\n") + %w[History.md LICENSE README.md] - s.homepage = "http://puma.io" + s.homepage = "https://puma.io" if s.respond_to?(:metadata=) s.metadata = { "bug_tracker_uri" => "https://github.com/puma/puma/issues", "changelog_uri" => "https://github.com/puma/puma/blob/master/History.md", - "homepage_uri" => "http://puma.io", + "homepage_uri" => "https://puma.io", "source_code_uri" => "https://github.com/puma/puma" } end diff --git a/test/config/t2_conf.rb b/test/config/t2_conf.rb new file mode 100644 index 0000000000..35533397c3 --- /dev/null +++ b/test/config/t2_conf.rb @@ -0,0 +1,3 @@ +log_requests +stdout_redirect "t2-stdout" +pidfile "t2-pid" diff --git a/test/helper.rb b/test/helper.rb index eda616b879..fa5cd20ca3 100644 --- a/test/helper.rb +++ b/test/helper.rb @@ -58,8 +58,11 @@ module TimeoutEveryTestCase class TestTookTooLong < Timeout::Error end - def run(*) - ::Timeout.timeout(RUBY_ENGINE == 'ruby' ? 60 : 120, TestTookTooLong) { super } + def time_it + t0 = Minitest.clock_time + ::Timeout.timeout(RUBY_ENGINE == 'ruby' ? 60 : 120, TestTookTooLong) { yield } + ensure + self.time = Minitest.clock_time - t0 end end diff --git a/test/helpers/ssl.rb b/test/helpers/ssl.rb index e9e7b24ea5..cfa5fec4ae 100644 --- a/test/helpers/ssl.rb +++ b/test/helpers/ssl.rb @@ -2,8 +2,8 @@ module SSLHelper def ssl_query @ssl_query ||= if Puma.jruby? @keystore = File.expand_path "../../../examples/puma/keystore.jks", __FILE__ - @ssl_cipher_list = "TLS_DHE_RSA_WITH_DES_CBC_SHA,TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA" - "keystore=#{@keystore}&keystore-pass=pswd&ssl_cipher_list=#{@ssl_cipher_list}" + @ssl_cipher_list = "TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" + "keystore=#{@keystore}&keystore-pass=blahblah&ssl_cipher_list=#{@ssl_cipher_list}" else @cert = File.expand_path "../../../examples/puma/cert_puma.pem", __FILE__ @key = File.expand_path "../../../examples/puma/puma_keypair.pem", __FILE__ diff --git a/test/shell/run.rb b/test/shell/run.rb index 58317f9293..458f640032 100644 --- a/test/shell/run.rb +++ b/test/shell/run.rb @@ -1,18 +1,10 @@ require "puma" require "puma/detect" -TESTS_TO_RUN = if Process.respond_to?(:fork) - %w[t2 t3] -else - %w[t2] -end - -results = TESTS_TO_RUN.map do |test| - system("ruby -rrubygems test/shell/#{test}.rb ") # > /dev/null 2>&1 -end +return unless Process.respond_to?(:fork) -if results.any? { |r| r != true } - exit 1 -else +if system("ruby -rrubygems test/shell/t3.rb ") exit 0 +else + exit 1 end diff --git a/test/shell/t2.rb b/test/shell/t2.rb deleted file mode 100644 index c76a8b2034..0000000000 --- a/test/shell/t2.rb +++ /dev/null @@ -1,19 +0,0 @@ -system "ruby -rrubygems -Ilib bin/pumactl -F test/shell/t2_conf.rb start &" - -sleep 1 until system "curl http://localhost:10103/" - -out=`ruby -rrubygems -Ilib bin/pumactl -F test/shell/t2_conf.rb status` - -system "ruby -rrubygems -Ilib bin/pumactl -F test/shell/t2_conf.rb stop" - -sleep 1 - -log = File.read("t2-stdout") - -File.unlink "t2-stdout" if File.file? "t2-stdout" - -if log =~ %r(GET / HTTP/1\.1) && !File.file?("t2-pid") && out == "Puma is started\n" - exit 0 -else - exit 1 -end diff --git a/test/shell/t2_conf.rb b/test/shell/t2_conf.rb deleted file mode 100644 index 40937207ba..0000000000 --- a/test/shell/t2_conf.rb +++ /dev/null @@ -1,5 +0,0 @@ -log_requests -stdout_redirect "t2-stdout" -pidfile "t2-pid" -bind "tcp://0.0.0.0:10103" -rackup File.expand_path('../rackup/hello.ru', File.dirname(__FILE__)) diff --git a/test/test_binder.rb b/test/test_binder.rb index 56eea2b8b6..2ca8f9e8f4 100644 --- a/test/test_binder.rb +++ b/test/test_binder.rb @@ -6,6 +6,7 @@ require "puma/binder" require "puma/puma_http11" require "puma/events" +require "puma/configuration" class TestBinderBase < Minitest::Test include SSLHelper @@ -15,6 +16,13 @@ def setup @binder = Puma::Binder.new(@events) end + def teardown + @binder.ios.reject! { |io| Minitest::Mock === io || io.to_io.closed? } + @binder.close + @binder.unix_paths.select! { |path| File.exist? path } + @binder.close_listeners + end + private def ssl_context_for_binder(binder = @binder) @@ -64,7 +72,7 @@ def test_home_alters_listeners_for_ssl_addresses def test_correct_zero_port @binder.parse ["tcp://localhost:0"], @events - m = %r!tcp://127.0.0.1:(\d+)!.match(@events.stdout.string) + m = %r!http://127.0.0.1:(\d+)!.match(@events.stdout.string) port = m[1].to_i refute_equal 0, port @@ -84,9 +92,9 @@ def test_correct_zero_port_ssl def test_logs_all_localhost_bindings @binder.parse ["tcp://localhost:0"], @events - assert_match %r!tcp://127.0.0.1:(\d+)!, @events.stdout.string + assert_match %r!http://127.0.0.1:(\d+)!, @events.stdout.string if Socket.ip_address_list.any? {|i| i.ipv6_loopback? } - assert_match %r!tcp://\[::1\]:(\d+)!, @events.stdout.string + assert_match %r!http://\[::1\]:(\d+)!, @events.stdout.string end end @@ -288,6 +296,34 @@ def test_socket_activation_unix File.unlink(path) rescue nil # JRuby race? end + def test_rack_multithread_default_configuration + binder = Puma::Binder.new(@events) + + assert binder.proto_env["rack.multithread"] + end + + def test_rack_multithread_custom_configuration + conf = Puma::Configuration.new(max_threads: 1) + + binder = Puma::Binder.new(@events, conf) + + refute binder.proto_env["rack.multithread"] + end + + def test_rack_multiprocess_default_configuration + binder = Puma::Binder.new(@events) + + refute binder.proto_env["rack.multiprocess"] + end + + def test_rack_multiprocess_custom_configuration + conf = Puma::Configuration.new(workers: 1) + + binder = Puma::Binder.new(@events, conf) + + assert binder.proto_env["rack.multiprocess"] + end + private def assert_activates_sockets(path: nil, port: nil, url: nil, sock: nil) @@ -315,13 +351,17 @@ def assert_parsing_logs_uri(order = [:unix, :tcp]) unix: "unix://test/#{name}_server.sock" } + expected_logs = prepared_paths.dup.tap do |logs| + logs[:tcp] = logs[:tcp].gsub('tcp://', 'http://') + end + tested_paths = [prepared_paths[order[0]], prepared_paths[order[1]]] @binder.parse tested_paths, @events stdout = @events.stdout.string order.each do |prot| - assert_match prepared_paths[prot], stdout + assert_match expected_logs[prot], stdout end ensure @binder.close_listeners if order.include?(:unix) && UNIX_SKT_EXIST @@ -331,7 +371,7 @@ def assert_parsing_logs_uri(order = [:unix, :tcp]) class TestBinderJRuby < TestBinderBase def test_binder_parses_jruby_ssl_options keystore = File.expand_path "../../examples/puma/keystore.jks", __FILE__ - ssl_cipher_list = "TLS_DHE_RSA_WITH_DES_CBC_SHA,TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA" + ssl_cipher_list = "TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" @binder.parse ["ssl://0.0.0.0:8080?#{ssl_query}"], @events diff --git a/test/test_cli.rb b/test/test_cli.rb index 22597f06f7..fa5a98f9a1 100644 --- a/test/test_cli.rb +++ b/test/test_cli.rb @@ -40,7 +40,8 @@ def test_control_for_tcp cntl = UniquePort.call url = "tcp://127.0.0.1:#{cntl}/" - cli = Puma::CLI.new [ "--control-url", url, + cli = Puma::CLI.new ["-b", "tcp://127.0.0.1:0", + "--control-url", url, "--control-token", "", "test/rackup/lobster.ru"], @events @@ -66,14 +67,14 @@ def test_control_for_tcp end def test_control_for_ssl - skip_on :jruby # Hangs on CI, TODO fix require "net/http" control_port = UniquePort.call control_host = "127.0.0.1" control_url = "ssl://#{control_host}:#{control_port}?#{ssl_query}" token = "token" - cli = Puma::CLI.new ["--control-url", control_url, + cli = Puma::CLI.new ["-b", "tcp://127.0.0.1:0", + "--control-url", control_url, "--control-token", token, "test/rackup/lobster.ru"], @events diff --git a/test/test_error_logger.rb b/test/test_error_logger.rb new file mode 100644 index 0000000000..1ca19ec20e --- /dev/null +++ b/test/test_error_logger.rb @@ -0,0 +1,70 @@ +require 'puma/error_logger' +require_relative "helper" + +class TestErrorLogger < Minitest::Test + Req = Struct.new(:env, :body) + + def test_stdio + error_logger = Puma::ErrorLogger.stdio + + assert_equal STDERR, error_logger.ioerr + end + + def test_info_with_only_error + _, err = capture_io do + Puma::ErrorLogger.stdio.info(error: StandardError.new('ready')) + end + + assert_match %r!#!, err + end + + def test_info_with_request + env = { + 'REQUEST_METHOD' => 'GET', + 'PATH_INFO' => '/debug', + 'HTTP_X_FORWARDED_FOR' => '8.8.8.8' + } + req = Req.new(env, '{"hello":"world"}') + + _, err = capture_io do + Puma::ErrorLogger.stdio.info(error: StandardError.new, req: req) + end + + assert_match %r!\("GET /debug" - \(8\.8\.8\.8\)\)!, err + end + + def test_info_with_text + _, err = capture_io do + Puma::ErrorLogger.stdio.info(text: 'The client disconnected while we were reading data') + end + + assert_match %r!The client disconnected while we were reading data!, err + end + + def test_debug_without_debug_mode + _, err = capture_io do + Puma::ErrorLogger.stdio.debug(text: 'blank') + end + + assert_empty err + end + + def test_debug_with_debug_mode + with_debug_mode do + _, err = capture_io do + Puma::ErrorLogger.stdio.debug(text: 'non-blank') + end + + assert_match %r!non-blank!, err + end + end + + private + + def with_debug_mode + original_debug, ENV["PUMA_DEBUG"] = ENV["PUMA_DEBUG"], "1" + yield + ensure + ENV["PUMA_DEBUG"] = original_debug + end +end diff --git a/test/test_events.rb b/test/test_events.rb index a749362330..0a1003d74b 100644 --- a/test/test_events.rb +++ b/test/test_events.rb @@ -1,3 +1,4 @@ +require 'puma/events' require_relative "helper" class TestEvents < Minitest::Test @@ -119,14 +120,16 @@ def test_error_writes_to_stderr_and_exits did_exit = false _, err = capture_io do - Puma::Events.stdio.error("interrupted") + begin + Puma::Events.stdio.error("interrupted") + rescue SystemExit + did_exit = true + ensure + assert did_exit + end end - assert_equal "ERROR: interrupted", err - rescue SystemExit - did_exit = true - ensure - assert did_exit + assert_match %r!ERROR: interrupted!, err end def test_pid_formatter @@ -175,7 +178,8 @@ def test_parse_error sock << "GET #{path}?a=#{params} HTTP/1.1\r\nConnection: close\r\n\r\n" sock.read sleep 0.1 # important so that the previous data is sent as a packet - assert_match %r!HTTP parse error, malformed request \(#{path}\)!, events.stderr.string + assert_match %r!HTTP parse error, malformed request!, events.stderr.string + assert_match %r!\("GET #{path}" - \(-\)\)!, events.stderr.string server.stop(true) end end diff --git a/test/test_http11.rb b/test/test_http11.rb index 267b8e8f17..d8c51e1e43 100644 --- a/test/test_http11.rb +++ b/test/test_http11.rb @@ -144,14 +144,14 @@ def test_max_uri_path_length parser = Puma::HttpParser.new req = {} - # Support URI path length to a max of 8196 + # Support URI path length to a max of 8192 path = "/" + rand_data(7000, 100) http = "GET #{path} HTTP/1.1\r\n\r\n" parser.execute(req, http, 0) assert_equal path, req['REQUEST_PATH'] parser.reset - # Raise exception if URI path length > 8196 + # Raise exception if URI path length > 8192 path = "/" + rand_data(9000, 100) http = "GET #{path} HTTP/1.1\r\n\r\n" assert_raises Puma::HttpParserError do diff --git a/test/test_integration_cluster.rb b/test/test_integration_cluster.rb index 6c0415052c..f409443c79 100644 --- a/test/test_integration_cluster.rb +++ b/test/test_integration_cluster.rb @@ -168,6 +168,20 @@ def test_refork refute_includes pids, get_worker_pids(1, WORKERS - 1) end + def test_fork_worker_spawn + cli_server '', config: <'/dev/null') + sleep 0.01 + exitstatus = Process.detach(pid).value.exitstatus + [200, {}, [exitstatus.to_s]] +end +RUBY + assert_equal '0', read_body(connect) + end + def test_nakayoshi cli_server "-w #{WORKERS} test/rackup/hello.ru", config: <(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -507,12 +509,15 @@ def test_chunked_request assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal "hello", body + assert_equal 5, content_length end def test_chunked_request_pause_before_value body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -525,12 +530,15 @@ def test_chunked_request_pause_before_value assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal "hello", body + assert_equal 5, content_length end def test_chunked_request_pause_between_chunks body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -543,12 +551,15 @@ def test_chunked_request_pause_between_chunks assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal "hello", body + assert_equal 5, content_length end def test_chunked_request_pause_mid_count body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -561,12 +572,15 @@ def test_chunked_request_pause_mid_count assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal "hello", body + assert_equal 5, content_length end def test_chunked_request_pause_before_count_newline body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -579,12 +593,15 @@ def test_chunked_request_pause_before_count_newline assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal "hello", body + assert_equal 5, content_length end def test_chunked_request_pause_mid_value body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -597,12 +614,15 @@ def test_chunked_request_pause_mid_value assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal "hello", body + assert_equal 5, content_length end def test_chunked_request_pause_between_cr_lf_after_size_of_second_chunk body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -624,12 +644,15 @@ def test_chunked_request_pause_between_cr_lf_after_size_of_second_chunk assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal (part1 + 'b'), body + assert_equal 4201, content_length end def test_chunked_request_pause_between_closing_cr_lf body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -643,12 +666,15 @@ def test_chunked_request_pause_between_closing_cr_lf assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal 'hello', body + assert_equal 5, content_length end def test_chunked_request_pause_before_closing_cr_lf body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -662,12 +688,15 @@ def test_chunked_request_pause_before_closing_cr_lf assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal 'hello', body + assert_equal 5, content_length end def test_chunked_request_header_case body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -675,12 +704,15 @@ def test_chunked_request_header_case assert_equal "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", data assert_equal "hello", body + assert_equal 5, content_length end def test_chunked_keep_alive body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -690,14 +722,17 @@ def test_chunked_keep_alive assert_equal ["HTTP/1.1 200 OK", "Content-Length: 0"], h assert_equal "hello", body + assert_equal 5, content_length sock.close end def test_chunked_keep_alive_two_back_to_back body = nil + content_length = nil server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] [200, {}, [""]] } @@ -715,6 +750,7 @@ def test_chunked_keep_alive_two_back_to_back h = header(sock) assert_equal ["HTTP/1.1 200 OK", "Content-Length: 0"], h assert_equal "hello", body + assert_equal 5, content_length assert_equal true, last_crlf_written last_crlf_writer.join @@ -726,16 +762,19 @@ def test_chunked_keep_alive_two_back_to_back assert_equal ["HTTP/1.1 200 OK", "Content-Length: 0"], h assert_equal "goodbye", body + assert_equal 7, content_length sock.close end def test_chunked_keep_alive_two_back_to_back_with_set_remote_address body = nil + content_length = nil remote_addr =nil @server = Puma::Server.new @app, @events, { remote_address: :header, remote_address_header: 'HTTP_X_FORWARDED_FOR'} server_run app: ->(env) { body = env['rack.input'].read + content_length = env['CONTENT_LENGTH'] remote_addr = env['REMOTE_ADDR'] [200, {}, [""]] } @@ -745,6 +784,7 @@ def test_chunked_keep_alive_two_back_to_back_with_set_remote_address h = header sock assert_equal ["HTTP/1.1 200 OK", "Content-Length: 0"], h assert_equal "hello", body + assert_equal 5, content_length assert_equal "127.0.0.1", remote_addr sock << "GET / HTTP/1.1\r\nX-Forwarded-For: 127.0.0.2\r\nConnection: Keep-Alive\r\nTransfer-Encoding: chunked\r\n\r\n4\r\ngood\r\n3\r\nbye\r\n0\r\n\r\n" @@ -754,6 +794,7 @@ def test_chunked_keep_alive_two_back_to_back_with_set_remote_address assert_equal ["HTTP/1.1 200 OK", "Content-Length: 0"], h assert_equal "goodbye", body + assert_equal 7, content_length assert_equal "127.0.0.2", remote_addr sock.close diff --git a/test/test_puma_server_ssl.rb b/test/test_puma_server_ssl.rb index bdd017dd15..584ab44f5c 100644 --- a/test/test_puma_server_ssl.rb +++ b/test/test_puma_server_ssl.rb @@ -11,10 +11,10 @@ class SSLEventsHelper < ::Puma::Events attr_accessor :addr, :cert, :error - def ssl_error(server, peeraddr, peercert, error) + def ssl_error(error, peeraddr, peercert) + self.error = error self.addr = peeraddr self.cert = peercert - self.error = error end end diff --git a/test/test_pumactl.rb b/test/test_pumactl.rb index 27d4d39254..96bafa28ec 100644 --- a/test/test_pumactl.rb +++ b/test/test_pumactl.rb @@ -150,7 +150,6 @@ def test_control_url_and_status end def test_control_ssl - skip_on :jruby # Hanging on JRuby, TODO fix host = "127.0.0.1" port = UniquePort.call url = "ssl://#{host}:#{port}?#{ssl_query}" diff --git a/test/test_redirect_io.rb b/test/test_redirect_io.rb index 7600aa16da..8a8119221d 100644 --- a/test/test_redirect_io.rb +++ b/test/test_redirect_io.rb @@ -17,7 +17,9 @@ def setup def teardown super - paths = [@out_file_path, @err_file_path, @old_out_file_path, @old_err_file_path].compact + paths = (skipped? ? [@out_file_path, @err_file_path] : + [@out_file_path, @err_file_path, @old_out_file_path, @old_err_file_path]).compact + File.unlink(*paths) @out_file = nil @err_file = nil diff --git a/test/test_thread_pool.rb b/test/test_thread_pool.rb index edb753fa91..0e6f7645ab 100644 --- a/test/test_thread_pool.rb +++ b/test/test_thread_pool.rb @@ -264,6 +264,21 @@ def test_shutdown_with_grace end assert_equal 0, pool.spawned assert_equal 2, rescued.length - refute rescued.any?(&:alive?) + refute rescued.compact.any?(&:alive?) + end + + def test_correct_waiting_count_for_killed_threads + pool = new_pool(1, 1) { |_| } + sleep 1 + + # simulate our waiting worker thread getting killed for whatever reason + pool.instance_eval { @workers[0].kill } + sleep 1 + pool.reap + sleep 1 + + pool << 0 + sleep 1 + assert_equal 0, pool.backlog end end