Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shared subscription in MQTT #355

Closed
4 of 6 tasks
oscgonfer opened this issue Sep 6, 2024 · 0 comments · Fixed by #361
Closed
4 of 6 tasks

Shared subscription in MQTT #355

oscgonfer opened this issue Sep 6, 2024 · 0 comments · Fixed by #361
Assignees

Comments

@oscgonfer
Copy link
Contributor

oscgonfer commented Sep 6, 2024

Check here: emqx/emqx#13566

TODO:

  • Two new "roles" of MQTT subscriber. "main" and "secondary", "main" subscribers have a persistent session, and "secondary" ones do not.
  • Update docker compose to allow horizontal scaling of both mqtt-tasks, and to provide a unique, but persistent client name somehow to MAIN subscribers. (is there a number in an an environment var or the hostname?). We need at least two "main" subscribers (as if both go down, we start losing messages). Secondaries can be added to and removed on demand.
  • Add Docker-compose healthchecks and restart policy for mqtt-task (how?)
  • Update mqtt task to pass a persistent client_id when subscribing, and to subscribe to to a shared subscription topic with THE SAME group eg $share/group/the/old/topic , setting clean_session to false on main subscribers, and to true on secondary.
  • Ensure that Main subscribers are ALWAYS bought up BEFORE secondaries, and shut down AFTER, in docker (how)? (tested, unnecessary)
  • Test extremely thoroughly on staging
  • Ptest should send N messages (where n is number of MAIN subscribers) and checks that they are ALL received
@oscgonfer oscgonfer added this to the Data Ingestion milestone Sep 6, 2024
@timcowlishaw timcowlishaw linked a pull request Sep 25, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants