From 697c9b231fe562539bbb3bcbebf92a009affc7a9 Mon Sep 17 00:00:00 2001 From: Noah <7409480+BitForger@users.noreply.github.com> Date: Wed, 3 Jul 2024 17:01:17 -0400 Subject: [PATCH 01/10] Create 24-07-glob-patterns.md --- proposals/24-07-glob-patterns.md | 166 +++++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 proposals/24-07-glob-patterns.md diff --git a/proposals/24-07-glob-patterns.md b/proposals/24-07-glob-patterns.md new file mode 100644 index 0000000000..fd07ecb56e --- /dev/null +++ b/proposals/24-07-glob-patterns.md @@ -0,0 +1,166 @@ + + +# Your short, descriptive title + +**Authors**: + +**Reviewers**: + +**Status**: implementable + + +## Summary + + + +## Motivation + + + +**Goals:** + +1. A goal +1. Another goal + + +**Non-goals:** + +1. A non-goal +1. Another non-goal + +## Proposal + + + +### User Stories + + +#### Story 1 + +Scenario summary: As a [end user, extension developer, ...], I want to [...] + + +#### Story 2 + +Scenario summary: As a [end user, extension developer, ...], I want to [...] + + +### Risks and Mitigations + + +### Dependencies + + +### Scalability + + +## Drawbacks + + +## Alternatives + + +## Rollout Plan + + +### Alpha + + +- Will the feature be gated by an "alpha" flag? Which one? +- Will the feature be available in `kubectl kustomize` during alpha? Why or why not? + +### Beta + + +### GA + From 6b9914e7b485f960629ef860ef38c6fe0efaf37c Mon Sep 17 00:00:00 2001 From: Noah <7409480+BitForger@users.noreply.github.com> Date: Wed, 3 Jul 2024 17:44:31 -0400 Subject: [PATCH 02/10] Update 24-07-glob-patterns.md --- proposals/24-07-glob-patterns.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/proposals/24-07-glob-patterns.md b/proposals/24-07-glob-patterns.md index fd07ecb56e..c91a3edd0e 100644 --- a/proposals/24-07-glob-patterns.md +++ b/proposals/24-07-glob-patterns.md @@ -11,11 +11,14 @@ To get started with this template: Ping `@kubernetes-sigs/kustomize-admins` and `@kubernetes-sigs/kustomize-maintainers`. --> -# Your short, descriptive title +# Add glob pattern support in the `resources` key of a user's `kustomization.yaml` -**Authors**: +**Authors**: +- BitForger **Reviewers**: +- natasha41575 +- KnVerey **Status**: implementable +File path globbing for the `resources` key enables users to maximize their use of Kustomize in their GitOps and CI/CD workflows. It simplifies their `kustomization.yaml` files by allowing them to pass a set of patterns that are equal to, or children of the `kustomization.yaml`'s working directory. ## Motivation +https://github.com/kubernetes-sigs/kustomize/issues/3205 is one of (at least two that I've seen) issues opened looking for this functionality. + +As operational tasks shift to a declarative approach, the need to apply a large set of manifests from a "source of truth" repository is increasingly common. Currently, Kustomize very tediously requires a user to define every individual path for each resource the user wishes to build and apply. When you have multiple sets of services you need to apply, this can very quickly spiral into `kustomization.yaml` files with a 100 or more lines of resources. File-path globbing was a feature in the past, but was removed, much to the communities chagrin. This proposal seeks to reverse that change. **Goals:** -1. A goal -1. Another goal +1. Resource resolution based on glob patterns +2. Resource resolution should only happen at the same or below the main `kustomization.yaml` file **Non-goals:** @@ -57,8 +64,7 @@ know that this has succeeded? What is out of scope for this proposal? Listing non-goals helps to focus discussion and make progress. --> -1. A non-goal -1. Another non-goal +1. Resource globbing should not have exclusions ## Proposal From 999016e50c5ed292dee83397ccfa0615882b3684 Mon Sep 17 00:00:00 2001 From: Noah <7409480+BitForger@users.noreply.github.com> Date: Wed, 3 Jul 2024 22:39:35 -0400 Subject: [PATCH 03/10] Update 24-07-glob-patterns.md --- proposals/24-07-glob-patterns.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/24-07-glob-patterns.md b/proposals/24-07-glob-patterns.md index c91a3edd0e..b4cbb18269 100644 --- a/proposals/24-07-glob-patterns.md +++ b/proposals/24-07-glob-patterns.md @@ -64,7 +64,7 @@ know that this has succeeded? What is out of scope for this proposal? Listing non-goals helps to focus discussion and make progress. --> -1. Resource globbing should not have exclusions +1. Resource globbing should not accept exclude patterns ## Proposal @@ -82,6 +82,8 @@ inline here, if you feel such implementation details are required to adequately If you have a PR, link to it at the top of this section. --> + + ### User Stories - +This can be implemented by adding a flag to the `kustomization.yaml` called `enableGlobbing` or `enableGlobSupport` that accepts a boolean value. When this flag is enabled, a user can pass glob patterns to the `resources` array and kustomize will take the pattern and collect all sibling and child paths that match the provided pattern. ### User Stories @@ -94,22 +94,69 @@ The goal here is to make this feel real for users without getting bogged down. #### Story 1 -Scenario summary: As a [end user, extension developer, ...], I want to [...] +Scenario summary: As a end user, I want to use a pattern to apply a bunch of manifests +Take the below file structure + +``` +. +└── k8s/ + ├── pipelines/ + │ ├── pipeline_1/ + │ │ ├── pipeline.yaml + │ │ ├── task.yaml + │ │ └── nested/ + │ │ └── task.yaml + │ └── pipeline_2/ + │ └── pipeline.yaml + ├── tasks/ + │ ├── shared_task_1.yaml + │ └── shared_task_2.yaml + └── kustomization.yaml +``` + +A user could pass the following to kustomize to apply all the matching files + +```yaml +resources: + - pipelines/**/*.yaml + - tasks/*/*.yaml +``` + +Kustomize would build all the matching files (using the already defined rules of kustomize, no name collisions, etc). + #### Story 2 -Scenario summary: As a [end user, extension developer, ...], I want to [...] +Scenario summary: As a end user, I want to apply a bunch of kustomizations using a pattern +``` +. +└── k8s/ + ├── bases/ + │ ├── app_1/ + │ │ ├── kustomization.yaml + │ │ ├── deployment.yaml + │ │ ├── service.yaml + │ │ └── ... + │ └── app_2/ + │ ├── kustomization.yaml + │ ├── deployment.yaml + │ ├── cronjob.yaml + │ └── ... + └── kustomization.yaml +``` + + ### Risks and Mitigations -This can be implemented by adding a flag to the `kustomization.yaml` called `enableGlobbing` or `enableGlobSupport` that accepts a boolean value. When this flag is enabled, a user can pass glob patterns to the `resources` array and kustomize will take the pattern and collect all sibling and child paths that match the provided pattern. +A feature flag will be added to the `kustomization.yaml` called `enableGlobbing` or `enableGlobSupport` that accepts a boolean value. When this flag is enabled, a user can pass glob patterns to the `resources` array and kustomize will take the pattern and collect all sibling and child paths that match the provided pattern. ### User Stories @@ -130,6 +130,12 @@ resources: Kustomize would build all the matching files (using the already defined rules of kustomize, no name collisions, etc). +Glob support should work on files names, so a pattern like this should work + +``` +tasks/*/shared_task_*.yaml +``` + #### Story 2 Scenario summary: As a end user, I want to apply a bunch of kustomizations using a pattern @@ -156,6 +162,12 @@ and config they'll use. └── kustomization.yaml ``` +A user could pass the following in their `kustomization.yaml` + +```yaml +resources: + - bases/**/ +``` ### Risks and Mitigations +There is the risk that a user could unintentionally include manifests if they choose to use the globbing support and they don't intentionally define their patterns to not be overly broad. For every other user, the file path globbing should be disabled by default so it shouldn't have an effect for them. + ### Dependencies +Go has a built in filepath package that should support the described use cases. + ### Scalability -Go has a built in filepath package that should support the described use cases. +Go has a built in filepath package that should be able to support the described use cases. ### Scalability +There may be a performance impact by using glob searching over looking up each individually named resource. These may scale with the amount of globs to lookup. I would have to do some testing to see how that behaved. + ## Drawbacks + + ## Alternatives +Currently, the only other approach is to use a script to re-generate the `kustomization.yaml` every time you go to apply a lot of manifests. This is unnecessary complexity for something many other ops tools support. + ## Rollout Plan -### Alpha +I don't believe this will need a formal graduation process as the proposal adds the feature behind an option flag in the `kustomization.yaml`. The defualt behavior of the `resources` key would not change. + + -### GA + -1. Resource resolution based on glob patterns -2. Resource resolution should only happen at the same or below the main `kustomization.yaml` file +1. Resource resolution based on glob patterns, at the sibling level and below the main `kustomization.yaml` file +2. **Non-goals:** From ea790405c3a30834ad0612686894eaf4e46d5601 Mon Sep 17 00:00:00 2001 From: Noah <7409480+BitForger@users.noreply.github.com> Date: Fri, 5 Jul 2024 22:10:03 -0400 Subject: [PATCH 09/10] Update 24-07-glob-patterns.md --- proposals/24-07-glob-patterns.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/24-07-glob-patterns.md b/proposals/24-07-glob-patterns.md index 287df684ce..581a094068 100644 --- a/proposals/24-07-glob-patterns.md +++ b/proposals/24-07-glob-patterns.md @@ -200,7 +200,7 @@ There may be a performance impact by using glob searching over looking up each i - +As discussed, this potentially reduces how explicit you are in defining the manifests you want to apply. However, the consensus seems to be that it's a worthwhile trade-off for the convenience added. ## Alternatives From 9c5bc003faf80c0f55455233cf4fc1165d5340c3 Mon Sep 17 00:00:00 2001 From: Noah <7409480+BitForger@users.noreply.github.com> Date: Fri, 5 Jul 2024 22:12:40 -0400 Subject: [PATCH 10/10] Update 24-07-glob-patterns.md --- proposals/24-07-glob-patterns.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/24-07-glob-patterns.md b/proposals/24-07-glob-patterns.md index 581a094068..7b23813a21 100644 --- a/proposals/24-07-glob-patterns.md +++ b/proposals/24-07-glob-patterns.md @@ -55,8 +55,8 @@ As operational tasks shift to a declarative approach, the need to apply a large List the specific goals of the proposal. What is it trying to achieve? How will we know that this has succeeded? --> -1. Resource resolution based on glob patterns, at the sibling level and below the main `kustomization.yaml` file -2. +1. Resource resolution based on glob patterns, at the sibling level and below of the main `kustomization.yaml` file + **Non-goals:**