Calling Redis from multiple fibers causes fibers to suspend indefinitely

[felixyz]

The bug is illustrated here:
https://gist.github.com/Felixyz/5771982

And a repo containing the same code and branches with variations here:
https://github.com/Felixyz/celluloid-redis-bug

• Reading from a Redis server using the redis gem with the Celluloid driver from multiple tasks in the same actor causes fibers to suspend and never be resumed, typically after 2 resumes (but in production code it's usually taken longer for this to happen, presumably because other processing intervenes, making the intervals between thread/fiber switching longer).
• Under JRuby, the same thing happens but consistently on all the fibers – not a single one is resumed. Update: This is not true. It's the behavior I saw earlier, but running the script repeatedly with JRuby yields a variety of results. At most, I've seen 6 out of 10 tasks resumed. The main point is that the bug appears similarly on MRI and JRuby (1.7.4).
• All the commands are received by the Redis server when on MRI, but not on JRuby, as illustrated by the gist.
• Using futures causes the same bug.
• Reading from separate actors (10 tasks 10 actors, as opposed to 10 tasks 1 actor) does not cause the bug.
• Sleeping or reading from a socket instead of calling the redis gem does not cause the bug. (This is illustrated in branches in the repo linked to above.)

[felixyz]

I've run the tests on OS X 10.8.2.

[felixyz]

I discussed this with @tarcieri on #celluloid and he pointed out that Redis can't handle interleaved requests on the same connection, so I guess this should just be closed.

Perhaps it might be interesting to add functionality to this gem to create an internal pool of connections? I guess it would have to be implicitly one pool per actor.

[tarcieri]

Another interesting approach would be a sort of "pipeline manager" that gives out Futures for requested commands, sends them to the server in a pipelined manner, and can match up these futures to Redis's responses. See:

http://redis.io/topics/pipelining