-
Notifications
You must be signed in to change notification settings - Fork 230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create and manage Tigris bucket for PG/Barman backups #3693
Conversation
If the bucket name is already taken, we should generate a unique name instead of erroring out |
OverrideName , it's automatically set to the target app name. I think the extension creation process in web will autogenerate random names but I don't see this easily exposed by flyctl's extension code. So in practice this means that PGs created with my-name will have buckets also named my-name .
I agree that the ideal case is not to bomb out on unique bucket names, but how likely is this in practice? I imagine:
The crash isn't ideal but it isn't entirely unexpected and easy enough to work around by removing the PG storage bucket. Either way, I don't want to spend much time overthinking this, so possible approaches if this needs to be addressed:
If this definitely needs addressing then I think I'd prefer 2, or 1 as long as we aren't pulling in another dependency. 2 is simple enough to just do and is probably a good idea anyway, so I'll just do that and we can decide what additional needs doing. |
Apparently somewhat, we hold onto recently-deleted bucket names so create/destroy/create cycles will fail even if the bucket itself is removed. Going to try scraping the error text for naming conflicts and appending a suffix, unless there's a better approach. |
Awesome start on this! Couple notes:
I think we should actually make this opt-in to start. There's a cost implication that some users may not be cool with.
It looks like this currently breaks if the bucket name doesn't match the App name.
|
…pped to arbitrary buckets. Try to fix, but currently non-functional.
Sorry, which log output do you mean specifically, and which additional secret am I injecting? Looks like I'm only injecting the one as per our docs. Thanks for the feedback. |
Agreed. Any thoughts about the easiest way to implement this? Version checks seem like the most immediately obvious, but that feels wrong and brittle. Seems like the better way is checking for the existence of this specific feature in the image, either with a command we can exec in and run, or something in the machine metadata. Thoughts? Unless of course there's something I'm missing with the version check, but semver version comparison feels like something we'd either need another dependency for or would have to be very careful about. |
There's already logic in place for this: |
…lain how they are enabled.
…ig to restored instance.
… Postgres clusters. Usage: fly postgres [command] Aliases: postgres, pg Available Commands: attach Attach a postgres cluster to an app barman Manage databases in a cluster config Show and manage Postgres configuration. connect Connect to the Postgres console create Create a new Postgres cluster db Manage databases in a cluster detach Detach a postgres cluster from an app events Track major cluster events failover Failover to a new primary import Imports database from a specified Postgres URI list List postgres clusters renew-certs Renews the SSH certificates for the Postgres cluster. restart Restarts each member of the Postgres cluster one by one. users Manage users in a postgres cluster Flags: -h, --help help for postgres Global Flags: -t, --access-token string Fly API Access Token --debug Print additional logs and traces --verbose Verbose output Use "fly postgres [command] --help" for more information about a command.
…'s using a custom imageRef.
} | ||
|
||
// Resolve the leader | ||
leader, err := pickLeader(ctx, machines) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will fail if the DB is having problems, or a host is down. What I did in ui-ex is spin up a new temporary machine in the same app - https://github.com/superfly/ui-ex/blob/add83100682486e9d9e8df2d9faf16a5e4a2342a/lib/fly/postgres.ex#L241
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good, just a few comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some command clarifications
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Change Summary
Integrates the new Tigris WAL storage into flyctl.
--enable-backup
flag tofly pg create
command to enable bucket/secret provisioning on cluster creation.--restore-target-app
,--restore-target-name
,--restore-target-time
and--restore-target-inclusive
flags tofly pg create
which behave as documented here. When a new instance with a cluster size of 1 is created with an app and optional target set, the specified backup is restored into the newly-created instance.fly pg backup enable
command for provisioning buckets and secrets into clusters where WAL backup is disabled.fly pg backup create
for creating backups, optionally named.fly pg backup list
to list existing backups.Related to:
Documentation