Skip to content
This repository has been archived by the owner on Dec 7, 2018. It is now read-only.

Calling Redis from multiple fibers causes fibers to suspend indefinitely #9

Open
felixyz opened this issue Jun 13, 2013 · 3 comments
Open

Comments

@felixyz
Copy link

felixyz commented Jun 13, 2013

The bug is illustrated here:
https://gist.github.com/Felixyz/5771982

And a repo containing the same code and branches with variations here:
https://github.com/Felixyz/celluloid-redis-bug

  • Reading from a Redis server using the redis gem with the Celluloid driver from multiple tasks in the same actor causes fibers to suspend and never be resumed, typically after 2 resumes (but in production code it's usually taken longer for this to happen, presumably because other processing intervenes, making the intervals between thread/fiber switching longer).
  • Under JRuby, the same thing happens but consistently on all the fibers – not a single one is resumed. Update: This is not true. It's the behavior I saw earlier, but running the script repeatedly with JRuby yields a variety of results. At most, I've seen 6 out of 10 tasks resumed. The main point is that the bug appears similarly on MRI and JRuby (1.7.4).
  • All the commands are received by the Redis server when on MRI, but not on JRuby, as illustrated by the gist.
  • Using futures causes the same bug.
  • Reading from separate actors (10 tasks 10 actors, as opposed to 10 tasks 1 actor) does not cause the bug.
  • Sleeping or reading from a socket instead of calling the redis gem does not cause the bug. (This is illustrated in branches in the repo linked to above.)
@felixyz
Copy link
Author

felixyz commented Jun 15, 2013

I've run the tests on OS X 10.8.2.

@felixyz
Copy link
Author

felixyz commented Jun 15, 2013

I discussed this with @tarcieri on #celluloid and he pointed out that Redis can't handle interleaved requests on the same connection, so I guess this should just be closed.

Perhaps it might be interesting to add functionality to this gem to create an internal pool of connections? I guess it would have to be implicitly one pool per actor.

@tarcieri
Copy link
Member

Another interesting approach would be a sort of "pipeline manager" that gives out Futures for requested commands, sends them to the server in a pipelined manner, and can match up these futures to Redis's responses. See:

http://redis.io/topics/pipelining

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants