Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tools): add yq #2406

Merged
merged 3 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/2-the-zarf-cli/100-cli-commands/zarf_tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,4 @@ Collection of additional tools to make airgap easier
* [zarf tools sbom](zarf_tools_sbom.md) - Generates a Software Bill of Materials (SBOM) for the given package
* [zarf tools update-creds](zarf_tools_update-creds.md) - Updates the credentials for deployed Zarf services. Pass a service key to update credentials for a single service
* [zarf tools wait-for](zarf_tools_wait-for.md) - Waits for a given Kubernetes resource to be ready
* [zarf tools yq](zarf_tools_yq.md) - yq is a lightweight and portable command-line data file processor.
81 changes: 81 additions & 0 deletions docs/2-the-zarf-cli/100-cli-commands/zarf_tools_yq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# zarf tools yq
<!-- Auto-generated by hack/gen-cli-docs.sh -->

yq is a lightweight and portable command-line data file processor.

## Synopsis

yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.

```
zarf tools yq [flags]
```

## Examples

```

# yq defaults to 'eval' command if no command is specified. See "zarf tools yq eval --help" for more examples.

# read the "stuff" node from "myfile.yml"
zarf tools yq '.stuff' < myfile.yml

# update myfile.yml in place
zarf tools yq -i '.stuff = "foo"' myfile.yml

# print contents of sample.json as idiomatic YAML
zarf tools yq -P sample.json

```

## Options

```
-C, --colors force print with colors
--csv-auto-parse parse CSV YAML/JSON values (default true)
--csv-separator char CSV Separator character (default ,)
-e, --exit-status set exit status if there are no matches or null or false is returned
--expression string forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.
--from-file string Load expression from specified file.
-f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
--header-preprocess Slurp any header comments and separators before processing expression. (default true)
-h, --help help for yq
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the file in place of first file given.
-p, --input-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|lua|l] parse format for input. (default "auto")
--lua-globals output keys as top-level global variables
--lua-prefix string prefix (default "return ")
--lua-suffix string suffix (default ";\n")
--lua-unquoted output unquoted string keys (e.g. {foo="bar"})
-M, --no-colors force print with no colors
-N, --no-doc Don't print document separators (---)
-0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
-o, --output-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|shell|s|lua|l] output format type. (default "auto")
-P, --prettyPrint pretty print, shorthand for '... style = ""'
--properties-array-brackets use [x] in array paths (e.g. for SpringBoot)
--properties-separator string separator to use between keys and values (default " = ")
-s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
--split-exp-file string Use a file to specify the split-exp expression.
--string-interpolation Toggles strings interpolation of \(exp) (default true)
--tsv-auto-parse parse TSV YAML/JSON values (default true)
-r, --unwrapScalar unwrap scalar, print the value with no quotes, colors or comments. Defaults to true for yaml (default true)
-v, --verbose verbose mode
-V, --version Print version information and quit
--xml-attribute-prefix string prefix for xml attributes (default "+@")
--xml-content-name string name for xml content (if no attribute name is present). (default "+content")
--xml-directive-name string name for xml directives (e.g. <!DOCTYPE thing cat>) (default "+directive")
--xml-keep-namespace enables keeping namespace after parsing attributes (default true)
--xml-proc-inst-prefix string prefix for xml processing instructions (e.g. <?xml version="1"?>) (default "+p_")
--xml-raw-token enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details. (default true)
--xml-skip-directives skip over directives (e.g. <!DOCTYPE thing cat>)
--xml-skip-proc-inst skip over process instructions (e.g. <?xml version="1"?>)
--xml-strict-mode enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.
```

## SEE ALSO

* [zarf tools](zarf_tools.md) - Collection of additional tools to make airgap easier
* [zarf tools yq eval](zarf_tools_yq_eval.md) - (default) Apply the expression to each document in each yaml file in sequence
* [zarf tools yq eval-all](zarf_tools_yq_eval-all.md) - Loads _all_ yaml documents of _all_ yaml files and runs expression once
91 changes: 91 additions & 0 deletions docs/2-the-zarf-cli/100-cli-commands/zarf_tools_yq_eval-all.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# zarf tools yq eval-all
<!-- Auto-generated by hack/gen-cli-docs.sh -->

Loads _all_ yaml documents of _all_ yaml files and runs expression once

## Synopsis

yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.

# Evaluate All ##
This command loads _all_ yaml documents of _all_ yaml files and runs expression once
Useful when you need to run an expression across several yaml documents or files (like merge).
Note that it consumes more memory than eval.


```
zarf tools yq eval-all [expression] [yaml_file1]... [flags]
```

## Examples

```

# Merge f2.yml into f1.yml (inplace)
zarf tools yq eval-all --inplace 'select(fileIndex == 0) * select(fileIndex == 1)' f1.yml f2.yml
# the same command and expression using shortened names:
zarf tools yq ea -i 'select(fi == 0) * select(fi == 1)' f1.yml f2.yml


# Merge all given files
zarf tools yq ea '. as $item ireduce ({}; . * $item )' file1.yml file2.yml ...

# Pipe from STDIN
# use '-' as a filename to pipe from STDIN
cat file2.yml | zarf tools yq ea '.a.b' file1.yml - file3.yml

```

## Options

```
-h, --help help for eval-all
```

## Options inherited from parent commands

```
-C, --colors force print with colors
--csv-auto-parse parse CSV YAML/JSON values (default true)
--csv-separator char CSV Separator character (default ,)
-e, --exit-status set exit status if there are no matches or null or false is returned
--expression string forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.
--from-file string Load expression from specified file.
-f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
--header-preprocess Slurp any header comments and separators before processing expression. (default true)
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the file in place of first file given.
-p, --input-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|lua|l] parse format for input. (default "auto")
--lua-globals output keys as top-level global variables
--lua-prefix string prefix (default "return ")
--lua-suffix string suffix (default ";\n")
--lua-unquoted output unquoted string keys (e.g. {foo="bar"})
-M, --no-colors force print with no colors
-N, --no-doc Don't print document separators (---)
-0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
-o, --output-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|shell|s|lua|l] output format type. (default "auto")
-P, --prettyPrint pretty print, shorthand for '... style = ""'
--properties-array-brackets use [x] in array paths (e.g. for SpringBoot)
--properties-separator string separator to use between keys and values (default " = ")
-s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
--split-exp-file string Use a file to specify the split-exp expression.
--string-interpolation Toggles strings interpolation of \(exp) (default true)
--tsv-auto-parse parse TSV YAML/JSON values (default true)
-r, --unwrapScalar unwrap scalar, print the value with no quotes, colors or comments. Defaults to true for yaml (default true)
-v, --verbose verbose mode
--xml-attribute-prefix string prefix for xml attributes (default "+@")
--xml-content-name string name for xml content (if no attribute name is present). (default "+content")
--xml-directive-name string name for xml directives (e.g. <!DOCTYPE thing cat>) (default "+directive")
--xml-keep-namespace enables keeping namespace after parsing attributes (default true)
--xml-proc-inst-prefix string prefix for xml processing instructions (e.g. <?xml version="1"?>) (default "+p_")
--xml-raw-token enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details. (default true)
--xml-skip-directives skip over directives (e.g. <!DOCTYPE thing cat>)
--xml-skip-proc-inst skip over process instructions (e.g. <?xml version="1"?>)
--xml-strict-mode enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.
```

## SEE ALSO

* [zarf tools yq](zarf_tools_yq.md) - yq is a lightweight and portable command-line data file processor.
93 changes: 93 additions & 0 deletions docs/2-the-zarf-cli/100-cli-commands/zarf_tools_yq_eval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# zarf tools yq eval
<!-- Auto-generated by hack/gen-cli-docs.sh -->

(default) Apply the expression to each document in each yaml file in sequence

## Synopsis

yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.

# Evaluate Sequence ##
This command iterates over each yaml document from each given file, applies the
expression and prints the result in sequence.

```
zarf tools yq eval [expression] [yaml_file1]... [flags]
```

## Examples

```

# Reads field under the given path for each file
zarf tools yq e '.a.b' f1.yml f2.yml

# Prints out the file
zarf tools yq e sample.yaml

# Pipe from STDIN
# use '-' as a filename to pipe from STDIN
cat file2.yml | zarf tools yq e '.a.b' file1.yml - file3.yml

# Creates a new yaml document
# Note that editing an empty file does not work.
zarf tools yq e -n '.a.b.c = "cat"'

# Update a file inplace
zarf tools yq e '.a.b = "cool"' -i file.yaml

```

## Options

```
-h, --help help for eval
```

## Options inherited from parent commands

```
-C, --colors force print with colors
--csv-auto-parse parse CSV YAML/JSON values (default true)
--csv-separator char CSV Separator character (default ,)
-e, --exit-status set exit status if there are no matches or null or false is returned
--expression string forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.
--from-file string Load expression from specified file.
-f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
--header-preprocess Slurp any header comments and separators before processing expression. (default true)
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the file in place of first file given.
-p, --input-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|lua|l] parse format for input. (default "auto")
--lua-globals output keys as top-level global variables
--lua-prefix string prefix (default "return ")
--lua-suffix string suffix (default ";\n")
--lua-unquoted output unquoted string keys (e.g. {foo="bar"})
-M, --no-colors force print with no colors
-N, --no-doc Don't print document separators (---)
-0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
-o, --output-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|shell|s|lua|l] output format type. (default "auto")
-P, --prettyPrint pretty print, shorthand for '... style = ""'
--properties-array-brackets use [x] in array paths (e.g. for SpringBoot)
--properties-separator string separator to use between keys and values (default " = ")
-s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
--split-exp-file string Use a file to specify the split-exp expression.
--string-interpolation Toggles strings interpolation of \(exp) (default true)
--tsv-auto-parse parse TSV YAML/JSON values (default true)
-r, --unwrapScalar unwrap scalar, print the value with no quotes, colors or comments. Defaults to true for yaml (default true)
-v, --verbose verbose mode
--xml-attribute-prefix string prefix for xml attributes (default "+@")
--xml-content-name string name for xml content (if no attribute name is present). (default "+content")
--xml-directive-name string name for xml directives (e.g. <!DOCTYPE thing cat>) (default "+directive")
--xml-keep-namespace enables keeping namespace after parsing attributes (default true)
--xml-proc-inst-prefix string prefix for xml processing instructions (e.g. <?xml version="1"?>) (default "+p_")
--xml-raw-token enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details. (default true)
--xml-skip-directives skip over directives (e.g. <!DOCTYPE thing cat>)
--xml-skip-proc-inst skip over process instructions (e.g. <?xml version="1"?>)
--xml-strict-mode enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.
```

## SEE ALSO

* [zarf tools yq](zarf_tools_yq.md) - yq is a lightweight and portable command-line data file processor.
Loading
Loading