[kwokctl] Support k8s format for snapshot #381
Conversation
k8s-ci-robot
added
do-not-merge/work-in-progress
✅ Deploy Preview for k8s-kwok canceled.
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: wzshiming The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
size/XL
k8s-ci-robot
added
the
kind/feature
wzshiming
force-pushed
the
feat/snapshot-yaml
branch
18 times, most recently
from
ce9d959 to
ea6f378
Compare
wzshiming
changed the title
wzshiming
force-pushed
the
feat/snapshot-yaml
branch
3 times, most recently
from
91116f9 to
7ef2f37
Compare
wzshiming
force-pushed
the
feat/snapshot-yaml
branch
12 times, most recently
from
e828375 to
08d503d
Compare
wzshiming
changed the title
k8s-ci-robot
removed
the
do-not-merge/work-in-progress
wzshiming
changed the title
wzshiming
force-pushed
the
feat/snapshot-yaml
branch
from
08d503d to
9ab2a0e
Compare
wzshiming
changed the title
wzshiming
force-pushed
the
feat/snapshot-yaml
branch
from
9ab2a0e to
2cbe85c
Compare
wzshiming
force-pushed
the
feat/snapshot-yaml
branch
from
2cbe85c to
5a9e9e4
Compare
|
|
||
| // Save saves the snapshot of cluster | ||
| func Save(ctx context.Context, rt Runtime, w io.Writer, filters []string) error { | ||
| err := rt.KubectlInCluster(exec.WithWriteTo(ctx, w), "get", strings.Join(filters, ","), "-o=yaml", "--all-namespaces") |
There was a problem hiding this comment.
in practice, it's likely to fail due to hitting QPS limits, right?
There was a problem hiding this comment.
I've tried the latest kubectl, which supports chunking without failing even with very large amounts of data.
However, the QPS limit causes it to be really slow, so I will try to remove the client/server QPS limit in the future to make this work better.
--chunk-size 500 Return large lists in chunks rather than all at once. Pass 0 to disable. This flag is beta and may change in the future.
There was a problem hiding this comment.
Let's put it to this week's meeting discussion. My understanding is that --chunk-size is not the same thing as "client-go's paged query utility" (where it can define pageSize and retry logic programmatically), so IMO the latter impl. is what we need to pursue.
There was a problem hiding this comment.
That's great, I have documented the relevant code links in advance as additional information for the discussion.
kubectl get seems to load all the resources into memory and then output them together, so too much memory is used and it fails.
There was a problem hiding this comment.
Yes, that is correct. In my local fork, I utilized client-go's paged query to take a snapshot of the API objects. I had planned to contribute it back for really a while... hopefully I can find some time to make it soon.
What type of PR is this?
/kind feature
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #32
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: