NSQ how to deal with different throughput queue chains?

[jannickj]

I have looked through all the documentation and this problem is not mentioned anywhere not even in the 5 situations it can fail

If I have a system of two queues that are dependent on each other and the first grows faster than the second. Then I risk running out of disk space... Is it assumed that I have infinite disk space? That is not the case in my setup.

The real problem occurs if both topics are on the same machine then I will eventually be unable to post to the second queue and hit a hard deadlock.

[jehiah]

@jannickj This is a good question for the mailing list nsq-users@googlegroups.com

NSQ is designed to be a distributed data stream which means that capacity constraints can be solved by scaling horizontally. It's outside the scope of NSQ to mange appropriate capacity for your specific workload.

[jannickj]

@jehiah Yes that would be outside its scope, however just being able to limit total queue size seemes like a faily simple solution.

[jehiah]

That feature is being tracked as #549