External backend for Locking.

[hden]

It would be a huge architectural improvement if we can turn the Atlantis server into a stateless container with external transactional storage for locks.

An abstract key-value store that implements the Backend interface seems to be a good start. We can start with a simple implementation such as the leveldb and the community can add more implementations.

Rationale

• A simple key-value store removes the requirement to manage the database schema.
• Any ACID-compatible database can be turned into a key-value store.
• Nowadays we have plenty of cloud providers for a hosted solution e.g. DynamoDB.

Benefits

• HA and autoscaling is easier to achieve with stateless architecture.
• Fits well with modern hosting option e.g. Knative / Cloud Run.
• If atlantis / Hashicorp are to turn Atlantis into a hosted service, it will be easier to provide a hosted key-value backend.

Related: #264

[kbiernat]

I just installed Atlantis and hit an edge case with that.
I'd like it to play nicely with the configured locking engine (such as DynamoDB), so that locks acquired by Atlantis also prevent an interference from another client.
For example, we have this Terraform host with Administrator permissions, that all TF used to be run (with file backend, and tfstate in the repo - don't ask..), and now I want to migrate to Atlantis, but still let people do TF stuff the old way. Is my only option to deny access to this host and only handle TF PRs via Atlantis, to avoid clashes?

[dmitri-lerko]

From my brief experiment with Cloud Run, there will be other challenges to address around eventing. Once Atlantis sends a response to GitHub's webhook, Cloud Run terminates atlantis container, which actually performs background processing. Therefore there is a need to launch Atlantis background processing via a separate Cloud Run endpoint/event before responding back to the GitHub.

[dinvlad]

Any updates/remaining interest in this one? Would be such a nice-to-have..

[mustafa89]

Would definitely be very welcome if this feature could be implemented.

[idlebot]

@dmitri-lerko, since late last year, it is possible to have an always-on Cloud Run container.

By the way, we currently run Atlantis on Google Cloud App Engine, but it is in my plans to migrate to Cloud Run in the future. As soon as I do, I'll post an update here.

[sahaqaa]

Would definitely be very welcome if this feature could be implemented.

[JonGilmore]

It's already implemented. See the updated docs, you can now use redis. This issue should be closed

[filipemiguelrocha]

@idlebot , have you migrated yet? Would like to know your experience.

[kishaningithub]

@JonGilmore @nitrocode Can the main documentation be also be updated to reflect this?
https://www.runatlantis.io/docs/deployment.html

[nitrocode]

Yes of course. This is an open source project and we welcome contributions. For anyone who would like to contribute, please see the runatlantis.io directory of this repo.