Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to using custom resource to store etcd snapshot metadata #8064

Merged
merged 14 commits into from
Oct 12, 2023

Commits on Oct 12, 2023

  1. Add ADR for etcd snapshot CRD migration

    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    b6fbf3b View commit details
    Browse the repository at this point in the history
  2. Minor updates as per design review discussion

    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    96188cf View commit details
    Browse the repository at this point in the history
  3. Add new CRD for etcd snapshots

    Also adds a hack go script to print the embedded CRDs, for developer use.
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    4583892 View commit details
    Browse the repository at this point in the history
  4. Move etcd snapshot code into separate file

    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    cb08a45 View commit details
    Browse the repository at this point in the history
  5. Elide old snapshot data when apiserver rejects configmap with ErrRequ…

    …estEntityTooLarge
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    69b549d View commit details
    Browse the repository at this point in the history
  6. Tidy s3 upload functions

    Consistently refer to object keys as such, simplify error handling.
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    df1cfc0 View commit details
    Browse the repository at this point in the history
  7. Consistently set snapshotFile timestamp

    Attempt to use timestamp from creation or filename instead of file/object modification times
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    af03eaf View commit details
    Browse the repository at this point in the history
  8. Move s3 snapshot list functionality to s3.go

    Also, don't list ONLY s3 snapshots if S3 is enabled.
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    29cfd48 View commit details
    Browse the repository at this point in the history
  9. Store extra metadata and cluster ID for snapshots

    Write the extra metadata both locally and to S3. These files are placed such that they will not be used by older versions of K3s that do not make use of them.
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    c574a77 View commit details
    Browse the repository at this point in the history
  10. Sort snapshots by time and key in tabwriter output

    Fixes snapshot list coming out in non-deterministic order
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    7c346a4 View commit details
    Browse the repository at this point in the history
  11. Move snapshot delete into local/s3 functions

    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    4d87b2c View commit details
    Browse the repository at this point in the history
  12. Switch to managing ETCDSnapshotFile resources

    Reconcile snapshot CRs instead of ConfigMap; manage ConfigMap downstream from CR list
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    58fb71e View commit details
    Browse the repository at this point in the history
  13. Add server token hash to CR and S3

    This required pulling the token hash stuff out of the cluster package, into util.
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    87cc6a2 View commit details
    Browse the repository at this point in the history
  14. Fix etcd snapshot integration tests

    Snapshot delete/prune tests were only working because the delete command
    would report success even when deleting a snapshot that didn't exist,
    and the test regex was finding the snapshot name multiple times in
    the list output and deleting it twice.
    
    Snapshot restore tests seem to have expected the deployment to be rolled out
    immediately, which is not a reasonable expectation.
    
    Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
    brandond committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    174ff79 View commit details
    Browse the repository at this point in the history