[Suggestion] Better performance with Erlang multisend? #12915
-
RabbitMQ series4.0.x Operating system (distribution) usedDebian How is RabbitMQ deployed?Community Docker image What would you like to suggest for a future version of RabbitMQ?There is a discussion and PR for multisend support in Erlang - being able to send the same message to many pids. Might help for broadcasting the same message to many clients. Do you think it is relevant? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
RabbitMQ uses the delegate module as custom solution to for example route a message to many queues. (The queues are hosted on a small number of different nodes. RabbitMQ clusters using Mnesia consist typically between 3 and 7 nodes). This custom solution has worked very well. |
Beta Was this translation helpful? Give feedback.
-
|
@benbro thank you for bringing this to our attention. Tanzu RabbitMQ had/has a prototype where it uses multiple TCP connections between nodes. It is very difficult to get right and the effects were not overwhelming, so the idea was dropped and hasn't resurfaced since. As David said, RabbitMQ has its own solution for passing around messages that must be routed to N queues on a remote replica. It has been around since 2011 or so. You can tune the number of these delegates to match the number of cores. It does its job well and it is very easy to reason about (compared to the prototype above). I did not read the entire discussion and parts of it focus on scheduler yielding, that is, node-local inter-process multi-cast. That's cheap compared to what For large fan-out scenarios you will likely be better off using a stream even then. |
Beta Was this translation helpful? Give feedback.
@benbro thank you for bringing this to our attention.
Tanzu RabbitMQ had/has a prototype where it uses multiple TCP connections between nodes. It is very difficult to get right and the effects were not overwhelming, so the idea was dropped and hasn't resurfaced since.
As David said, RabbitMQ has its own solution for passing around messages that must be routed to N queues on a remote replica. It has been around since 2011 or so. You can tune the number of these delegates to match the number of cores. It does its job well and it is very easy to reason about (compared to the prototype above).
I did not read the entire discussion and parts of it focus on scheduler yielding, that is, node-loca…