Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(resource): don't use -f when patch file is provided #13317

Merged
merged 21 commits into from
Jul 27, 2024

Conversation

agilgur5
Copy link

@agilgur5 agilgur5 commented Jul 6, 2024

Fixes #13279 per #13279 (reply in thread)
Fixes #11248 and Supersedes #11249

Motivation

When using kubectl patch:

  1. If a patch file is provided (not a full manifest), then you have to use -p and provide a kind and name in flags. You cannot use -f in this case.
  2. If a full manifest is provided, then you have to use both -p and -f
    • in this form, -f is expected to have a header containing the kind and name (and potentially more)
    • and -p is the patch file
    • in our case, only one manifest is provided that acts as both: a patch file with a header

Modifications

  • don't add -f if there are flags and a kind is provided in the manifest
  • use --patch-file instead of -p, no need to stringify
    • (--patch-file has existed since at least k8s 1.20. we import the kubectl package though, so it only depends on the version we use in go.mod)
  • update unit tests
  • update & add examples, which are used as part of the test-examples E2E suite

Verification

Unit tests and E2E tests (the examples are tested)

- `-f` and `-p` are mutually exclusive, regardless of `strategy`
  - we can replace both with `--patch-file` instead

Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Anton Gilgur added 3 commits July 6, 2024 16:32
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
@agilgur5

This comment was marked as resolved.

Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
@agilgur5
Copy link
Author

agilgur5 commented Jul 6, 2024

but this could use an E2E, and I don't think one exists yet for resource templates?

Ah, the example tests actually take care of this. Unfortunately they are failing:

Running examples/k8s-patch-basic.yaml...
workflow.argoproj.io/k8s-patch-basic-vfsbq created
error: timed out waiting for the condition on workflows/k8s-patch-basic-vfsbq
NAME                    STATUS    AGE   MESSAGE
k8s-patch-basic-vfsbq   Running   30s   

The error lacks some detail though:

    State:          Terminated
      Reason:       Error
      Exit Code:    1

and there are no logs in CI because only the wait container logs are retrieved, and a resource template has no wait container. Will PR a fix shortly to get logs of all containers in the failure debug, which should be helpful for other purposes as well. EDIT: Fixed in #13318

@agilgur5
Copy link
Author

agilgur5 commented Jul 6, 2024

and there are no logs

Here's the error running locally:

time="2024-07-06T22:55:25.552Z" level=info msg="kubectl patch --type strategic --patch-file /tmp/manifest.yaml -o json"
error: You must provide one or more resources by argument or filename.
Example resource specifications include:
   '-f rsrc.yaml'
   '--filename=rsrc.json'
   '<resource> <name>'
   '<resource>'

In other words, --patch-file only works if a resource has been specified. So this does require that flags are specified, meaning that we do have to do some inference to determine whether -f or --patch-file should be used, as they are not equivalent.

Also noting that the json patch example, which has flags, works correctly

Anton Gilgur added 11 commits July 6, 2024 19:08
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
@agilgur5 agilgur5 changed the title fix(resource): use --patch-file for patches fix(resource): don't use -f when patch file is provided Jul 7, 2024
Anton Gilgur added 4 commits July 7, 2024 14:28
- effectively, they are supposed to be two different files, one is just a header (`-f`) that contains `name` and `kind` (etc) and the other is a patch file
  - in our case though, only one manifest is provided, so use it as both, assuming that it includes a header

- and refactor and fix some other logic
  - JSON patch can fail unmarshaling, as it can be an array instead of a map
  - fix pointer issues (classic)

Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
@agilgur5 agilgur5 marked this pull request as ready for review July 7, 2024 22:41
@agilgur5 agilgur5 requested a review from terrytangyuan July 7, 2024 22:42
@agilgur5 agilgur5 added this to the v3.5.x patches milestone Jul 7, 2024
name: "strategic -f --patch-file",
patchType: "strategic",
appendFileFlag: true,
manifestPath: "../../examples/hello-world.yaml", // any YAML with a `kind`
Copy link
Member

@Joibel Joibel Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather this (and the other manifestPaths under test here) were specific to the test suite, not overloading other files which have their own purpose. The effect of changing or deleting one of these (albeit unlikely) and seeing this test failing would be very surprising.

Copy link
Author

@agilgur5 agilgur5 Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but the existing tests already did that, so I did not change that test behavior in this PR. Changing that seemed to require a more substantial refactor, as we don't have a testdata dir for unit tests (this requires a file to read in) or much use of full manifests in units, so I deferred that for the purposes of this fix PR.

I could make a separate PR for that, but we'd have to think of a strategy to apply more generically to these types of unit tests that a need a testdata dir or similar

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that's reasonable.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jul 26, 2024
Signed-off-by: Anton Gilgur <4970083+agilgur5@users.noreply.github.com>
@agilgur5 agilgur5 enabled auto-merge (squash) July 27, 2024 16:57
@agilgur5 agilgur5 merged commit 5d8ee22 into argoproj:main Jul 27, 2024
28 checks passed
@agilgur5 agilgur5 deleted the fix-resource-patch-file-flag branch July 27, 2024 16:59
agilgur5 added a commit that referenced this pull request Jul 30, 2024
Signed-off-by: Anton Gilgur <agilgur5@gmail.com>
Signed-off-by: Anton Gilgur <4970083+agilgur5@users.noreply.github.com>
(cherry picked from commit 5d8ee22)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/templates/resource lgtm This PR has been approved by a maintainer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Resource patch fails if mergeStrategy: merge or strategic and patch file with flags provided
2 participants