Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recommend default queuing backend for Active Job #25

Open
ellery-nava opened this issue Jun 5, 2024 · 3 comments · May be fixed by #47
Open

Recommend default queuing backend for Active Job #25

ellery-nava opened this issue Jun 5, 2024 · 3 comments · May be fixed by #47
Assignees

Comments

@ellery-nava
Copy link
Contributor

What's the task?

Active Job comes out of the box with an in-memory queueing system. However, this is not sufficient for a production system - because the queue only exists in memory, pending tasks will be lost if the system restarts.

This ticket is to investigate the pros and cons of various queuing systems and recommend a default backend that will work well for most new Nava projects.

@ellery-nava
Copy link
Contributor Author

@rocketnova @lorenyu @SammySteiner - starting to take a look at some of the recommended libraries from the documentation.... I assume that backends that require Redis or another queue service not currently a part of the infrastructure would be less attractive than something like Good Job that runs using Postgres?

Any thoughts on recommending a single library, vs. doing a short writeup on pros/cons of a couple options? Wondering whether something like Resque, for example, might become more attractive if a project is already planning on using Redis.

@rocketnova
Copy link
Contributor

@ellery-nava I think a short writeup on pros/cons sounds very helpful, especially in terms of considerations that teams should account for when selecting a backend.

Licensing and hosting costs should definitely be a consideration. I'd say that it's possible for something that is Postgres-backed to be more expensive than something Redis-backed, but I think it really depends. From a quick search, it sounds like ElastiCache for Redis is cheaper but really best used as a caching layer, rather than as the primary data store, whereas MemoryDB for Redis is great as the primary data store, but really expensive.

@lorenyu
Copy link

lorenyu commented Jun 14, 2024

if needed SQS-backed is also ok. Azure has an equivalent.

Postgres-backed would be easiest since that wouldn't be different between AWS and Azure

@ellery-nava ellery-nava linked a pull request Jun 27, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants