Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cmd/bundle): add new bundle build sub-commands #2334

Merged
merged 20 commits into from
Nov 8, 2023

Conversation

GeorgeMac
Copy link
Contributor

@GeorgeMac GeorgeMac commented Nov 3, 2023

Fixes FLI-653
Fixes FLI-657

This PR adds the following subcommands:

flipt bundle build [REFERENCE]

flipt bundle list
DIGEST    REPO       TAG      CREATED
0eb282e   readonly   latest   2023-11-03 17:14:31 +0000 UTC
3e628b6   readonly            2023-11-03 15:54:54 +0000 UTC

The build command builds a target directory containing flipt feature state into an OCI bundle.
The target reference can be a local bundle name (e.g. example:latest) or even a remote target (e.g. some.registry.io/example:latest).

The list command lists out existing local bundles.

It also includes JSON support for import and export.

Copy link
Contributor

github-actions bot commented Nov 3, 2023

Uffizzi Preview deployment-40330 was deleted.

Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple linter issues

internal/oci/file.go Show resolved Hide resolved
Copy link

codecov bot commented Nov 3, 2023

Codecov Report

Merging #2334 (fc59ef6) into gm/fs-oci (e5fe37c) will decrease coverage by 0.19%.
The diff coverage is 67.55%.

@@              Coverage Diff              @@
##           gm/fs-oci    #2334      +/-   ##
=============================================
- Coverage      70.85%   70.66%   -0.19%     
=============================================
  Files             80       81       +1     
  Lines           7679     7892     +213     
=============================================
+ Hits            5441     5577     +136     
- Misses          1918     1977      +59     
- Partials         320      338      +18     
Files Coverage Δ
internal/config/storage.go 86.86% <ø> (+4.29%) ⬆️
internal/ext/exporter.go 86.32% <100.00%> (-0.07%) ⬇️
internal/ext/importer.go 73.16% <100.00%> (ø)
internal/storage/fs/oci/source.go 76.92% <75.00%> (+0.45%) ⬆️
internal/ext/common.go 88.00% <91.66%> (+3.38%) ⬆️
internal/ext/encoding.go 86.66% <86.66%> (ø)
internal/storage/fs/snapshot.go 71.63% <72.85%> (+1.34%) ⬆️
internal/oci/file.go 58.68% <59.66%> (-8.62%) ⬇️

... and 1 file with indirect coverage changes

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

Base automatically changed from gm/fs-oci-source to gm/fs-oci November 3, 2023 18:20
Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, any chance we could improve test coverage a bit before merging?

internal/ext/common.go Show resolved Hide resolved
internal/storage/fs/snapshot.go Show resolved Hide resolved
@markphelps markphelps added wip Work In Progress needs docs Requires documentation updates labels Nov 8, 2023
Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to add output to the build command that it expects a ref as an argument?

It doesn't seem clear to me from the defaults that cobra give us:

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build
Error: accepts 1 arg(s), received 0
Usage:
  flipt bundle build [flags]

Flags:
  -h, --help   help for build

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build --help
Build a bundle

Usage:
  flipt bundle build [flags]

Flags:
  -h, --help   help for build

@@ -87,6 +88,7 @@ func (c *exportCommand) run(cmd *cobra.Command, _ []string) error {
// default to stdout
out io.Writer = os.Stdout
logger = zap.Must(zap.NewDevelopment())
enc = ext.EncodingYML
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we give the user the ability to set this format via a CLI flag now that we support JSON? could default to yaml still

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh yeah I wondered about this. So it will support JSON if you specify a file with .json as the target.
However, for STDOUT, yeah we could have an actual flag. Same for import and STDIN.

@markphelps
Copy link
Collaborator

markphelps commented Nov 8, 2023

Is it possible to add output to the build command that it expects a ref as an argument?

It doesn't seem clear to me from the defaults that cobra give us:

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build
Error: accepts 1 arg(s), received 0
Usage:
  flipt bundle build [flags]

Flags:
  -h, --help   help for build

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build --help
Build a bundle

Usage:
  flipt bundle build [flags]

Flags:
  -h, --help   help for build

Also wondering if we want to make it ignore our own /internal directory which will bundle up all the test features.yaml when we are developing inside Flipt itself?

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build flags:latest
2023-11-08T09:24:14-05:00	DEBUG	configuration source	{"path": "/Users/markphelps/Library/Application Support/flipt/config.yml"}
2023-11-08T09:24:14-05:00	DEBUG	index file does not exist, defaulting...	{"file": ".flipt.yml", "error": "open .flipt.yml: no such file or directory"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/git/testdata/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "7f897ae61d6ac2cbb860432fb9ecc2be7734323ee6614c4c3a14f4f1d639aba9", "namespace": "production"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/local/testdata/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "7f897ae61d6ac2cbb860432fb9ecc2be7734323ee6614c4c3a14f4f1d639aba9", "namespace": "production"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/s3/testdata/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "7f897ae61d6ac2cbb860432fb9ecc2be7734323ee6614c4c3a14f4f1d639aba9", "namespace": "production"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/s3/testdata/prefix/prefix_features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "702592274c467b2a219c6e43a07de1e9b3556f28c7d00c4283f849c4ec8d1a36", "namespace": "prefix"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/invalid/boolean_flag_segment/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "fb260e56d5433b2ddcb0c4f6d81a8dc9631208062028b61a865a139663f9779d", "namespace": "fruit"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/invalid/variant_flag_distribution/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "54aeae045c47a82c81f4e7864cca73bfb1b340561340afb13449c3656a0d127c", "namespace": "fruit"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/invalid/variant_flag_segment/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "8901690e170db22dc97ccbccf1f474bb98b37a9425a5c9bcf3521fa8de444ed7", "namespace": "fruit"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/empty_features/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/prod/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/prod/prod.features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "79829852d38165912ebee90bb57798a65ea3c755c1a02ca0d1429591872dd8b8", "namespace": "production"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/sandbox/features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/sandbox/sandbox.features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "9498e44d48c05d0efaea8484c7dacadc439474f3d85f1226fd53220b53f3d8ff", "namespace": "sandbox"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/staging/features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/staging/staging.features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/prod/features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "50631f69883631c74db7a9a44ea90385a04753a6cee4b442273fb82bd23a34d1", "namespace": "production"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/prod/prod.features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "79829852d38165912ebee90bb57798a65ea3c755c1a02ca0d1429591872dd8b8", "namespace": "production"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "10838f68c638eca90edc37d39aff52beb8d6849aaac43961fbd7379c72bcc2f6", "namespace": "staging"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/sandbox/features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "2df6f34d50983731af5fca913d0d73a0af6452068e936f35eba656d0f4028bfa", "namespace": "sandbox"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/sandbox/sandbox.features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "9498e44d48c05d0efaea8484c7dacadc439474f3d85f1226fd53220b53f3d8ff", "namespace": "sandbox"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/staging.features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "126afd90914805e061401d19c44cc4331fe0ad01fb7e3fd952688b6ba931dbfb", "namespace": "staging"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/yaml_stream/features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "cfa38b386ce87874e2bc227e48d94ce0cd6ae062bdeeb90276bf5d12a4a2a8a5", "namespace": "fruit"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "55f2eb9a0383abf46ecd5bbb731ebca6a7792866f36b691c8be65fa453837023", "namespace": "football"}

@GeorgeMac
Copy link
Contributor Author

Is it possible to add output to the build command that it expects a ref as an argument?
It doesn't seem clear to me from the defaults that cobra give us:

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build
Error: accepts 1 arg(s), received 0
Usage:
  flipt bundle build [flags]

Flags:
  -h, --help   help for build

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build --help
Build a bundle

Usage:
  flipt bundle build [flags]

Flags:
  -h, --help   help for build

Also wondering if we want to make it ignore our own /internal directory which will bundle up all the test features.yaml when we are developing inside Flipt itself?

workspace/flipt - [gm/bundles-build] » ./bin/flipt bundle build flags:latest
2023-11-08T09:24:14-05:00	DEBUG	configuration source	{"path": "/Users/markphelps/Library/Application Support/flipt/config.yml"}
2023-11-08T09:24:14-05:00	DEBUG	index file does not exist, defaulting...	{"file": ".flipt.yml", "error": "open .flipt.yml: no such file or directory"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/git/testdata/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "7f897ae61d6ac2cbb860432fb9ecc2be7734323ee6614c4c3a14f4f1d639aba9", "namespace": "production"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/local/testdata/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "7f897ae61d6ac2cbb860432fb9ecc2be7734323ee6614c4c3a14f4f1d639aba9", "namespace": "production"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/s3/testdata/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "7f897ae61d6ac2cbb860432fb9ecc2be7734323ee6614c4c3a14f4f1d639aba9", "namespace": "production"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/s3/testdata/prefix/prefix_features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "702592274c467b2a219c6e43a07de1e9b3556f28c7d00c4283f849c4ec8d1a36", "namespace": "prefix"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/invalid/boolean_flag_segment/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "fb260e56d5433b2ddcb0c4f6d81a8dc9631208062028b61a865a139663f9779d", "namespace": "fruit"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/invalid/variant_flag_distribution/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "54aeae045c47a82c81f4e7864cca73bfb1b340561340afb13449c3656a0d127c", "namespace": "fruit"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/invalid/variant_flag_segment/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	adding layer	{"digest": "8901690e170db22dc97ccbccf1f474bb98b37a9425a5c9bcf3521fa8de444ed7", "namespace": "fruit"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/empty_features/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/prod/features.yml"}
2023-11-08T09:24:14-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/prod/prod.features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "79829852d38165912ebee90bb57798a65ea3c755c1a02ca0d1429591872dd8b8", "namespace": "production"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/sandbox/features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/sandbox/sandbox.features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "9498e44d48c05d0efaea8484c7dacadc439474f3d85f1226fd53220b53f3d8ff", "namespace": "sandbox"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/staging/features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/explicit_index/staging/staging.features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/prod/features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "50631f69883631c74db7a9a44ea90385a04753a6cee4b442273fb82bd23a34d1", "namespace": "production"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/prod/prod.features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "79829852d38165912ebee90bb57798a65ea3c755c1a02ca0d1429591872dd8b8", "namespace": "production"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "10838f68c638eca90edc37d39aff52beb8d6849aaac43961fbd7379c72bcc2f6", "namespace": "staging"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/sandbox/features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "2df6f34d50983731af5fca913d0d73a0af6452068e936f35eba656d0f4028bfa", "namespace": "sandbox"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/sandbox/sandbox.features.yml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "9498e44d48c05d0efaea8484c7dacadc439474f3d85f1226fd53220b53f3d8ff", "namespace": "sandbox"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/implicit_index/staging/staging.features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "126afd90914805e061401d19c44cc4331fe0ad01fb7e3fd952688b6ba931dbfb", "namespace": "staging"}
2023-11-08T09:24:15-05:00	DEBUG	opening state file	{"path": "internal/storage/fs/testdata/valid/yaml_stream/features.yaml"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "cfa38b386ce87874e2bc227e48d94ce0cd6ae062bdeeb90276bf5d12a4a2a8a5", "namespace": "fruit"}
2023-11-08T09:24:15-05:00	DEBUG	adding layer	{"digest": "55f2eb9a0383abf46ecd5bbb731ebca6a7792866f36b691c8be65fa453837023", "namespace": "football"}

Do you know which files you would expect to be included from the root of the Flipt repo itself?
We could either add them to a .flipt.yml explicitly, or we can add an exclude directive for internal.

@markphelps
Copy link
Collaborator

Yeah maybe an exclude directive for internal makes sense?

Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀🌕

@GeorgeMac GeorgeMac merged commit 08213a5 into gm/fs-oci Nov 8, 2023
25 of 27 checks passed
@GeorgeMac GeorgeMac deleted the gm/bundles-build branch November 8, 2023 18:28
@markphelps markphelps removed the needs docs Requires documentation updates label Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wip Work In Progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants