Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow chart deploy overrides ALPHA #2403

Merged
merged 14 commits into from
Apr 9, 2024
Merged
65 changes: 65 additions & 0 deletions adr/0023-chart-override.md
naveensrinivasan marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# 23. Simplifying Helm Chart Value Overrides

Date: 2024-03-25

## Status

Accepted


## Context

The process of deploying applications with Helm charts in Kubernetes environments often necessitates the customization of chart values to align with specific operational or environmental requirements. The current method for customizing these values—either through manual edits or `###ZARF_VAR_XYZ###`. A more streamlined approach would greatly enhance the deployment experience by offering both flexibility and reliability.

## Decision

To address this issue, we propose the introduction of a feature designed to simplify the process of overriding chart values at the time of deployment. This feature would allow users to easily specify overrides for any chart values directly via command-line arguments, eliminating the need to alter the chart's default values file or manage multiple command-line arguments for each override.

Key aspects of the proposed implementation include:
- Use existing `--set` flags to specify overrides for chart values.
- The ability to list all overrides in a structured and easily understandable format within `zarf.yaml`.
- Ensuring that during deployment, these specified overrides take precedence over the chart's default values, thus facilitating customized deployments without necessitating permanent modifications to the chart.

## Consequences

Adopting this feature would lead to several key improvements:
- **Streamlined Configuration Process**: Allowing helm values overrides in the zarf package schema simplifies the user experience by reducing the reliance on extensive custom `###ZARF_VAR_XYZ###` templating and aligning more closely with standard Helm practices

Ultimately, this feature is aimed at enhancing the deployment workflow by offering a straightforward and efficient means of customizing Helm chart deployments via command-line inputs.

## Example Configuration

Below is an example of how the `zarf.yaml` configuration file might be structured to utilize the new override feature for Helm chart values:

```yaml
kind: ZarfPackageConfig
metadata:
name: helm-charts
description: Example showcasing multiple ways to deploy helm charts
version: 0.0.1

components:
- name: demo-helm-charts
required: true
charts:
- name: podinfo-local
version: 6.4.0
namespace: podinfo-from-local-chart
localPath: chart
valuesFiles:
- values.yaml
variables:
- name: REPLICA_COUNT
description: "Override the number of pod replicas"
path: replicaCount
```
This configuration allows for the specification of default values and descriptions for variables that can be overridden at deployment time. The variables section under each chart specifies the variables that can be overridden, along with a path that indicates where in the values file the variable is located.

### Command Line Example

To override the `REPLICA_COUNT` variable at deployment time, the following command can be used:

```bash
zarf package deploy zarf-package-helm-charts-arm64-0.0.1.tar.zst --set REPLICA_COUNT=5
```
This command demonstrates how users can easily customize their Helm chart deployments by specifying overrides for chart values directly via command-line arguments, in line with the proposed feature.
121 changes: 105 additions & 16 deletions docs/3-create-a-zarf-package/4-zarf-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -1185,6 +1185,95 @@ Must be one of:
</blockquote>
</details>

<details open>
<summary>
<strong> <a name="components_items_charts_items_variables"></a>variables</strong>
</summary>
&nbsp;
<blockquote>

## components > charts > variables

**Description:** [alpha] List of variables to set in the Helm chart

| | |
| -------- | ------- |
| **Type** | `array` |

![Min Items: N/A](https://img.shields.io/badge/Min%20Items%3A%20N/A-gold)
![Max Items: N/A](https://img.shields.io/badge/Max%20Items%3A%20N/A-gold)
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_13"></a>ZarfChartVariable

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
| **Type** | `object` |
| **Additional properties** | [![Not allowed](https://img.shields.io/badge/Not%20allowed-red)](# "Additional Properties not allowed.") |
| **Defined in** | #/definitions/ZarfChartVariable |

<details>
<summary>
<strong> <a name="components_items_charts_items_variables_items_name"></a>name *</strong>
</summary>
&nbsp;
<blockquote>

![Required](https://img.shields.io/badge/Required-red)

**Description:** The name of the variable

| | |
| -------- | -------- |
| **Type** | `string` |

| Restrictions | |
| --------------------------------- | ----------------------------------------------------------------------------- |
| **Must match regular expression** | ```^[A-Z0-9_]+$``` [Test](https://regex101.com/?regex=%5E%5BA-Z0-9_%5D%2B%24) |

</blockquote>
</details>

<details>
<summary>
<strong> <a name="components_items_charts_items_variables_items_description"></a>description *</strong>
</summary>
&nbsp;
<blockquote>

![Required](https://img.shields.io/badge/Required-red)

**Description:** A brief description of what the variable controls

| | |
| -------- | -------- |
| **Type** | `string` |

</blockquote>
</details>

<details>
<summary>
<strong> <a name="components_items_charts_items_variables_items_path"></a>path *</strong>
</summary>
&nbsp;
<blockquote>

![Required](https://img.shields.io/badge/Required-red)

**Description:** The path within the Helm chart values where this variable applies

| | |
| -------- | -------- |
| **Type** | `string` |

</blockquote>
</details>

</blockquote>
</details>

</blockquote>
</details>

Expand All @@ -1208,7 +1297,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_13"></a>ZarfDataInjection
### <a name="autogenerated_heading_14"></a>ZarfDataInjection

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -1371,7 +1460,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_14"></a>ZarfFile
### <a name="autogenerated_heading_15"></a>ZarfFile

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -1465,7 +1554,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_15"></a>symlinks items
### <a name="autogenerated_heading_16"></a>symlinks items

| | |
| -------- | -------- |
Expand Down Expand Up @@ -1511,7 +1600,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_16"></a>images items
### <a name="autogenerated_heading_17"></a>images items

| | |
| -------- | -------- |
Expand All @@ -1538,7 +1627,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_17"></a>repos items
### <a name="autogenerated_heading_18"></a>repos items

| | |
| -------- | -------- |
Expand Down Expand Up @@ -1633,7 +1722,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_18"></a>valuesFiles items
### <a name="autogenerated_heading_19"></a>valuesFiles items

| | |
| -------- | -------- |
Expand Down Expand Up @@ -1676,7 +1765,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_19"></a>fluxPatchFiles items
### <a name="autogenerated_heading_20"></a>fluxPatchFiles items

| | |
| -------- | -------- |
Expand Down Expand Up @@ -1824,7 +1913,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_20"></a>env items
### <a name="autogenerated_heading_21"></a>env items

| | |
| -------- | -------- |
Expand Down Expand Up @@ -1939,7 +2028,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_21"></a>ZarfComponentAction
### <a name="autogenerated_heading_22"></a>ZarfComponentAction

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -2029,7 +2118,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_22"></a>env items
### <a name="autogenerated_heading_23"></a>env items

| | |
| -------- | -------- |
Expand Down Expand Up @@ -2094,7 +2183,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_23"></a>ZarfComponentActionSetVariable
### <a name="autogenerated_heading_24"></a>ZarfComponentActionSetVariable

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -2448,7 +2537,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_24"></a>ZarfComponentAction
### <a name="autogenerated_heading_25"></a>ZarfComponentAction

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -2479,7 +2568,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_25"></a>ZarfComponentAction
### <a name="autogenerated_heading_26"></a>ZarfComponentAction

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -2510,7 +2599,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_26"></a>ZarfComponentAction
### <a name="autogenerated_heading_27"></a>ZarfComponentAction

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -2590,7 +2679,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_27"></a>ZarfPackageConstant
### <a name="autogenerated_heading_28"></a>ZarfPackageConstant

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -2709,7 +2798,7 @@ Must be one of:
![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)

### <a name="autogenerated_heading_28"></a>ZarfPackageVariable
### <a name="autogenerated_heading_29"></a>ZarfPackageVariable

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
Expand Down
7 changes: 7 additions & 0 deletions examples/helm-charts/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ components:
localPath: chart
valuesFiles:
- values.yaml
# Variables are used to override the default values in the chart
# This can be overridden by the user at deployment time with the `--set` flag
variables:
- name: REPLICA_COUNT
description: "Override the number of pod replicas"
path: replicaCount

- name: podinfo-oci
version: 6.4.0
Expand Down Expand Up @@ -77,3 +83,4 @@ components:
name: cool-release-name-podinfo
namespace: podinfo-from-repo
condition: available

53 changes: 45 additions & 8 deletions src/pkg/packager/deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -547,24 +547,61 @@ func (p *Packager) pushReposToRepository(reposPath string, repos []string) error
return nil
}

// generateValuesOverrides creates a map containing overrides for chart values based on the given inputs.
// It prioritizes the overrides in this order:
// 1. Variables defined in the ZarfComponent, including both Set Variables and Chart Variables.
// 2. Deployment options specified through DeployOpts.
// This function is useful for customizing the deployment of a Helm chart by programmatically setting chart values.
//
// Returns:
// - A map containing the final set of value overrides for the chart, where keys are the variable names.
func generateValuesOverrides(chartVariables []types.ZarfChartVariable,
setVariableMap map[string]*types.ZarfSetVariable,
deployOpts types.ZarfDeployOptions,
componentName, chartName string) (map[string]any, error) {

valuesOverrides := make(map[string]any)
chartOverrides := make(map[string]any)

for _, variable := range chartVariables {
if setVar, ok := setVariableMap[variable.Name]; ok && setVar != nil {
// Use the variable's path as a key to ensure unique entries for variables with the same name but different paths.
if err := helpers.MergePathAndValueIntoMap(chartOverrides, variable.Path, setVar.Value); err != nil {
return nil, fmt.Errorf("unable to merge path and value into map: %w", err)
}
}
}

// Apply any direct overrides specified in the deployment options for this component and chart
if componentOverrides, ok := deployOpts.ValuesOverridesMap[componentName]; ok {
if chartSpecificOverrides, ok := componentOverrides[chartName]; ok {
valuesOverrides = chartSpecificOverrides
}
}

// Merge chartOverrides into valuesOverrides to ensure all overrides are applied.
// This corrects the logic to ensure that chartOverrides and valuesOverrides are merged correctly.
return helpers.MergeMapRecursive(valuesOverrides, chartOverrides), nil
}

// Install all Helm charts and raw k8s manifests into the k8s cluster.
func (p *Packager) installChartAndManifests(componentPaths *layout.ComponentPaths, component types.ZarfComponent) (installedCharts []types.InstalledChart, err error) {
for _, chart := range component.Charts {

// zarf magic for the value file
for idx := range chart.ValuesFiles {
chartValueName := helm.StandardValuesName(componentPaths.Values, chart, idx)
if err := p.valueTemplate.Apply(component, chartValueName, false); err != nil {
return installedCharts, err
}
}

// TODO (@WSTARR): Currently this logic is library-only and is untested while it is in an experimental state - it may eventually get added as shorthand in Zarf Variables though
var valuesOverrides map[string]any
if componentChartValuesOverrides, ok := p.cfg.DeployOpts.ValuesOverridesMap[component.Name]; ok {
if chartValuesOverrides, ok := componentChartValuesOverrides[chart.Name]; ok {
valuesOverrides = chartValuesOverrides
}
// this is to get the overrides for the chart from the commandline and
// the chart variables set in the ZarfComponent and as well as DeployOpts set from the Library user.
// The order of precedence is as follows:
// 1. Set Variables and Chart Variables from the ZarfComponent
// 2. DeployOpts
valuesOverrides, err := generateValuesOverrides(chart.Variables, p.cfg.SetVariableMap, p.cfg.DeployOpts, component.Name, chart.Name)
if err != nil {
return installedCharts, err
}

helmCfg := helm.New(
Expand Down
Loading
Loading