Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outbox Sweeper: Support circuit breaking of Topics #3155

Open
preardon opened this issue Jun 17, 2024 · 0 comments
Open

Outbox Sweeper: Support circuit breaking of Topics #3155

preardon opened this issue Jun 17, 2024 · 0 comments
Assignees

Comments

@preardon
Copy link
Member

preardon commented Jun 17, 2024

The way in which the outbox sweeper works is to grab the top x dispatched messages and attempt a dispatch every y number of seconds. This problem with this is if the top x messages all fail to dispatch (due to a issue with the Topic / configuration / ect.) then the sweeper will continue to just dispatch the same x messages and fail repeatedly stopping any other messages from dispatching.

Describe the solution you'd like
If there is an issue dispatching for a particular topic I would it to be removed from the next x number of attempts to dispatch i.e. if there are 200 messages to TopicA followed 100 messages to TopicB, the dispatcher is set to 100 at at time and there is an issue dispatching to TopicA, on the next tick of the sweeper I would like any messages for TopicA to be ignored and the messages from TopicB to be dispatched

Additional context
I have had issues with Topic filling up (1gb) on Azure Service Bus as well as permissions not being correctly applied which have caused the outbox sweeper to essentially stall as it continually tries to dispatch the same un-dispatchable messages

@preardon preardon changed the title Outbox Sw: Support circuit breaking of Topics Outbox Sweeper: Support circuit breaking of Topics Jun 17, 2024
@preardon preardon self-assigned this Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant