Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ConnectionResetError on Pipeline calls #1202

Closed
FodderMK opened this issue Aug 13, 2019 · 3 comments
Closed

ConnectionResetError on Pipeline calls #1202

FodderMK opened this issue Aug 13, 2019 · 3 comments

Comments

@FodderMK
Copy link

FodderMK commented Aug 13, 2019

Version:
redis-py: 3.3.6
redis server: 4.0.12

Platform:
Linux 4.14.94+ x86_64 (Google Cloud Memorystore)

Description:
After upgrading from v3.2.1 to 3.3.6 I started to get ConnectionResetError exceptions on longer-running instances of my app.

  [...]
  File "/env/local/lib/python3.7/site-packages/redis/client.py", line 3671, in execute
    self.shard_hint)
  File "/env/local/lib/python3.7/site-packages/redis/connection.py", line 1065, in get_connection
    if connection.can_read():
  File "/env/local/lib/python3.7/site-packages/redis/connection.py", line 682, in can_read
    return self._parser.can_read(timeout)
  File "/env/local/lib/python3.7/site-packages/redis/connection.py", line 295, in can_read
    return self._buffer and self._buffer.can_read(timeout)
  File "/env/local/lib/python3.7/site-packages/redis/connection.py", line 205, in can_read
    raise_on_timeout=False)
  File "/env/local/lib/python3.7/site-packages/redis/connection.py", line 173, in _read_from_socket
    data = recv(self._sock, socket_read_size)
  File "/env/local/lib/python3.7/site-packages/redis/_compat.py", line 58, in recv
    return sock.recv(*args, **kwargs)
ConnectionResetError: [Errno 104] Connection reset by peer

I'm initializing my connection with health checks:

redis_client = redis.Redis(
  host=host,
  port=port,
  socket_timeout=10,
  socket_keepalive=True,
  socket_connect_timeout=10,
  retry_on_timeout=True,
  health_check_interval=10,
)

p = redis_client.pipeline()
[...]
result = p.execute()

From what I can tell the path for execute() does a sock.recv() on a stale connection before it tries to do the health check, causing the exception to occur first. Should execute be trying a health check first?

@andymccurdy
Copy link
Contributor

andymccurdy commented Aug 13, 2019

The problem here is that socket.error (or in this case, a subclass of socket.error) is being raised rather than a redis.exceptions.ConnectionError. This translation was lost during the 3.3.0 changes. ConnectionErrors (in the case of testing the health of the socket as this is trying to do) are automatically resolved by the ConnectionPool.

I've re-added the translation and have released 3.3.7. If for some reason you're still seeing issues, please re-open this issue.

@FodderMK
Copy link
Author

I'm not seeing these errors any more. Thanks for the fix!

@andymccurdy
Copy link
Contributor

Great, thanks for reporting back!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants