Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor snapshot for format k8s #450

Merged
merged 1 commit into from
Apr 7, 2023
Merged

Refactor snapshot for format k8s #450

merged 1 commit into from
Apr 7, 2023

Conversation

wzshiming
Copy link
Member

@wzshiming wzshiming commented Apr 4, 2023 •
edited
Loading

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #32
Refactor for #381 (comment)

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 4, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wzshiming

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from yibozhuang April 4, 2023 12:12
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 4, 2023
@netlify
Copy link

netlify bot commented Apr 4, 2023 •
edited
Loading

✅ Deploy Preview for k8s-kwok canceled.

Name Link
🔨 Latest commit 2099e55
🔍 Latest deploy log https://app.netlify.com/sites/k8s-kwok/deploys/643032c21438050008622758

@wzshiming wzshiming force-pushed the refactor/snapshot-k8s branch 6 times, most recently from ffbd11c to c1cd7da Compare April 4, 2023 15:56
@wzshiming wzshiming requested a review from Huang-Wei April 6, 2023 02:13
@wzshiming wzshiming force-pushed the refactor/snapshot-k8s branch 2 times, most recently from 1b05c31 to e90b7ba Compare April 6, 2023 06:20
@Huang-Wei
Copy link
Contributor

Thanks @wzshiming. I will review tomorrow.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 6, 2023
@wzshiming wzshiming force-pushed the refactor/snapshot-k8s branch 2 times, most recently from 0eccc73 to e3d6251 Compare April 6, 2023 07:12
Huang-Wei
Huang-Wei reviewed Apr 6, 2023
View reviewed changes
Copy link
Contributor

@Huang-Wei Huang-Wei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good start to implement snapshot save --format=k8s and snapshot export. Some perf optimizations and customizations can be implemented iteratively.

pkg/kwokctl/snapshot/save.go Outdated Show resolved Hide resolved
pkg/kwokctl/snapshot/save.go
err = result.Visit(func(info *resource.Info, err error) error {
gvrs := make([]schema.GroupVersionResource, 0, len(resources))
for _, resource := range resources {
mapping, err := mappingFor(restMapper, resource)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line is hitting the server has received too many requests and has asked us to try again later every time when I tested it locally against a 160-node cluster.

I suppose the code is borrowed from cli-runtime which seems to have some perf issue if you want to iterate mapping a series of resources. We may turn to construct the GVRs locally.

Copy link
Contributor

@Huang-Wei Huang-Wei Apr 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another topic is the log is not revealing where this error message comes from, which makes debugging a bit inconvenient.

pkg/kwokctl/cmd/snapshot/export/export.go
type flagpole struct {
Path string
Kubeconfig string
Filters []string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future, we may seek a fine-grained interface via componentconfig, so it can:

  • adapt to lower-version k8s clusters as which may employ different GVK (like v1beta1 runtimeclass vs. v1)
  • fine-grained behavior on fields stripping (like, whether or not strip a pod's nodeName, some particular annotations, security contexts, controllerRef, finalizers, etc.)
  • ......

pkg/kwokctl/snapshot/load.go Outdated Show resolved Hide resolved
pkg/kwokctl/snapshot/load.go Outdated Show resolved Hide resolved
site/content/en/docs/generated/kwokctl_snapshot.md Outdated Show resolved Hide resolved
site/content/en/docs/user/kwokctl-snapshot.md Outdated Show resolved Hide resolved
site/content/en/docs/user/kwokctl-snapshot.md Outdated Show resolved Hide resolved
site/content/en/docs/user/kwokctl-snapshot.md Outdated Show resolved Hide resolved
site/content/en/docs/user/kwokctl-snapshot.md Show resolved Hide resolved
@wzshiming wzshiming force-pushed the refactor/snapshot-k8s branch 7 times, most recently from a3de6c9 to 7a75e60 Compare April 7, 2023 03:01
@wzshiming wzshiming changed the title Refactor snapshot for format k8s [WIP] Refactor snapshot for format k8s Apr 7, 2023
@wzshiming wzshiming force-pushed the refactor/snapshot-k8s branch 4 times, most recently from c4512b4 to 36221da Compare April 7, 2023 10:14
@wzshiming wzshiming changed the title [WIP] Refactor snapshot for format k8s Refactor snapshot for format k8s Apr 7, 2023
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 7, 2023
@wzshiming wzshiming force-pushed the refactor/snapshot-k8s branch 16 times, most recently from 7f67af2 to 3c42b6b Compare April 7, 2023 12:39
@wzshiming wzshiming force-pushed the refactor/snapshot-k8s branch from 3c42b6b to 2099e55 Compare April 7, 2023 15:12
@Huang-Wei
Copy link
Contributor

/lgtm

Thanks @wzshiming!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 7, 2023
@Huang-Wei
Copy link
Contributor

Created #456 to track follow-ups.

@k8s-ci-robot k8s-ci-robot merged commit 244efa2 into kubernetes-sigs:main Apr 7, 2023
@wzshiming wzshiming deleted the refactor/snapshot-k8s branch April 8, 2023 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Huang-Wei Huang-Wei Huang-Wei left review comments

@yibozhuang yibozhuang Awaiting requested review from yibozhuang

Assignees

@Huang-Wei Huang-Wei

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[kwokctl] Support k8s format for snapshot
3 participants
@wzshiming @k8s-ci-robot @Huang-Wei