nsqd: persist on disk when in-flight (optionally?)

[tj]

totally understand that it's not a 1.0 thing but for us it's a limiting factor right now. we have to basically have excess machines just for the memory capacity, not a huge huge deal but it would be nice if they could remain on disk (at least the payload)

[jehiah]

There are two in-memory priority queues where messages are kept. One handles when messages are in-flight to consumers so we can time them out appropriately, and the other is when they are requeued and are deferred until a certain time.

#34 covers the desire for a disk backed priority queue implementation so we are not memory bound by messages in those states, so i think it covers this ask.

[mreiferson]

yea, this is covered by #34, my current opinion is that I'd like to see an implementation that simply journals all messages to disk by default. The priority queues would just contain metadata.