Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prefer IO#wait_readable/IO#wait_writable rather than IO.select. #960

Merged
merged 3 commits into from Nov 17, 2020

Conversation

ioquatix
Copy link
Contributor

This allows the Ruby 3 scheduler hooks to improve concurrency of Redis.

Fixes #959

This allows the Ruby 3 scheduler hooks to improve concurrency of Redis.
@byroot
Copy link
Collaborator

byroot commented Nov 16, 2020

I can't find any documentation on these. In which version were they introduced?

Also it seems like some IOs don't have these methods, like CI shows:

NoMethodError: undefined method `wait_readable' for #<Redis::Connection::SSLSocket:0x0000559ec0a576c0>

@byroot
Copy link
Collaborator

byroot commented Nov 16, 2020

Oh nevermind, I found them. It's part of the io/wait extension: http://ruby-doc.com/stdlib-2.3.0_preview1/libdoc/io/wait/rdoc/IO.html

@ioquatix
Copy link
Contributor Author

Redis::Connection::SSLSocket is not a normal IO so it should be delegating to the OpenSSL::SSL::SSLSocket instance I guess. I'll need to check it.

The method has been around since Ruby 2.0 at least: https://docs.ruby-lang.org/en/2.0.0/IO.html#method-i-wait_readable

@ioquatix
Copy link
Contributor Author

Hmmmm... It looks like SSLSocket does not expose wait_readable or wait_writable but we can delegate to the underlying I/O.

However, I think we should add this method to OpenSSL gem... I'll coordinate that to happen. It will probably be the same implementation as this PR so there shouldn't be any issue.

@byroot
Copy link
Collaborator

byroot commented Nov 16, 2020

I think we should add this method to OpenSSL gem.

Sounds like it indeed.

In the meantime I suppose we can feature check and use IO.select as a fallback?

@ioquatix
Copy link
Contributor Author

It's very trivial to delegate to the underlying I/O which is completely correct so there should be no need for IO.select.

@ioquatix
Copy link
Contributor Author

I also noticed this:

          begin
            # Initiate the socket connection in the background. If it doesn't fail
            # immediately it will raise an IO::WaitWritable (Errno::EINPROGRESS)
            # indicating the connection is in progress.
            # Unlike waiting for a tcp socket to connect, you can't time out ssl socket
            # connections during the connect phase properly, because IO.select only partially works.
            # Instead, you have to retry.
            ssl_sock.connect_nonblock
          rescue Errno::EAGAIN, Errno::EWOULDBLOCK, IO::WaitReadable
            if ssl_sock.wait_readable(timeout)
              retry
            else
              raise TimeoutError
            end
          rescue IO::WaitWritable
            if ssl_sock.wait_writable(timeout)
              retry
            else
              raise TimeoutError
            end
          end

I don't have time to investigate WHY this was a problem previously, but I don't know if the retry is needed or not.

@ioquatix
Copy link
Contributor Author

Do you think you can cut a release after merging this?

@byroot
Copy link
Collaborator

byroot commented Nov 16, 2020

I don't have time to investigate WHY

It's late here, and I don't quite remember the details but:

@ioquatix
Copy link
Contributor Author

Thanks for the links, if it's a bug with OpenSSL we should definitely investigate further.

@byroot
Copy link
Collaborator

byroot commented Nov 16, 2020

Do you think you can cut a release after merging this?

Sure, but I'd like to make sure we didn't regress on the SSL thing first. And I'll probably batch it with a couple other fixes.

@ioquatix
Copy link
Contributor Author

connect_nonblock(exception: false)

is supported in OpenSSL gem... but not sure how far back it goes (compatibility).

@ioquatix
Copy link
Contributor Author

Okay, looks like all tests are passing - let me know if you need anything else :)

@byroot
Copy link
Collaborator

byroot commented Nov 16, 2020

let me know if you need anything else

Should be good. Also I noticed we actually added a unit test for the SSL hanging issue, so sounds good to me. I'll merge tomorrow morning.

@byroot byroot merged commit ed22638 into redis:master Nov 17, 2020
@ioquatix ioquatix deleted the prefer-io-wait branch November 17, 2020 08:37
koic added a commit to koic/rubocop that referenced this pull request Sep 11, 2021
…er` cop

Fixes rubocop#9061.

This PR adds new `Lint/IncompatibleIoSelectWithFiberScheduler` cop.
It checks for `IO.select` that is incompatible with Fiber Scheduler since Ruby 3.0.

```ruby
# bad
IO.select([io], [], [], timeout)

# good
io.wait_readable(timeout)

# bad
IO.select([], [io], [], timeout)

# good
io.wait_writable(timeout)
```

Ref: `Fiber Scheduler` section of https://www.ruby-lang.org/en/news/2020/12/25/ruby-3-0-0-released/

This PR will make it possible to detect proven cases with redis/redis-rb#960.
bbatsov pushed a commit to rubocop/rubocop that referenced this pull request Sep 12, 2021
Fixes #9061.

This PR adds new `Lint/IncompatibleIoSelectWithFiberScheduler` cop.
It checks for `IO.select` that is incompatible with Fiber Scheduler since Ruby 3.0.

```ruby
# bad
IO.select([io], [], [], timeout)

# good
io.wait_readable(timeout)

# bad
IO.select([], [io], [], timeout)

# good
io.wait_writable(timeout)
```

Ref: `Fiber Scheduler` section of https://www.ruby-lang.org/en/news/2020/12/25/ruby-3-0-0-released/

This PR will make it possible to detect proven cases with redis/redis-rb#960.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Avoid using IO.select
2 participants