lock-exec
is a CLI that makes it easy to run at-most-once commands in a distributed environment. At Loom we run multiple identical Kubernetes clusters and we use lock-exec
to ensure that Kubernetes cron jobs only run in a single cluster. lock-exec
uses dynamodb to lock on a user specified key, run the input command, and then unlock the key.
You must already be authenticated to AWS with a default region. lock-exec
uses config.LoadDefaultConfig
to load AWS credentials using the standard credential chain and does not currently support any direct method of authentication.
Additionally, lock-exec
requires a dynamodb table to use for locking. This table must have a partition key named key
that stores the lock keys. The default table name is lock-exec
. If you use a different table name you must specify it explictly using the --table
flag.
The following IAM permissions are required on the DynamoDB table containing the locks:
dynamodb:GetItem
dynamodb:PutItem
dynamodb:UpdateItem
dynamodb:DeleteItem
AWS CLI
aws dynamodb create-table --table-name lock-exec \
--table-class STANDARD \
--billing-mode PAY_PER_REQUEST \
--key-schema AttributeName=key,KeyType=HASH
--attribute-definitions AttributeName=key,AttributeType=S
Terraform
resource "aws_dynamodb_table" "lock_exec" {
name = "lock-exec"
table_class = "STANDARD"
billing_mode = "PAY_PER_REQUEST"
hash_key = "key"
attribute {
name = "key"
type = "S"
}
}
Download the lock-exec
binary from the releases page. You can also install using Go or Docker.
# Using Go
go install github.com/loomhq/lock-exec@latest
# Using Docker
docker run ghcr.io/loomhq/lock-exec --help
Basic usage is lock-exec run <key> <command>
.
$ go run main.go run testkey 'echo "hello world"' -t loomctl-locks
{"level":"info","ts":1651558192.655782,"caller":"cmd/run.go:22","msg":"running command","key":"testkey","command":"echo \"hello world\""}
{"level":"info","ts":1651558192.944402,"caller":"cmd/run.go:33","msg":"command succeeded","key":"testkey","command":"echo \"hello world\"","output":"\"hello world\"\n"}
Once the command finishes running lock-exec
will unlock the key. It also listens for os interrupts and unlocks the key before exiting. In the rare case where lock-exec
exits and fails to unlock the key will remain locked for the next 24 hours (you can customize this with --expire <duration>
). The key can manually be unlocked earlier using lock-exec unlock <key>
.