Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(cli): Support for separate write repo #1447

Closed
wants to merge 3 commits into from
Closed

feat(cli): Support for separate write repo #1447

wants to merge 3 commits into from

Conversation

mihaigalos
Copy link

@mihaigalos mihaigalos commented Feb 1, 2024 •
edited
Loading

This PR adds a new field for specifying a write repo destination.
If the field is present, it is consumed. If not, the repoURL is consumed instead (same behaviour as before).

Potential impl for #1354.

@netlify Netlify
Copy link

netlify bot commented Feb 1, 2024 •
edited
Loading

✅ Deploy Preview for docs-kargo-akuity-io ready!

Name Link
🔨 Latest commit 08feb00
🔍 Latest deploy log https://app.netlify.com/sites/docs-kargo-akuity-io/deploys/65bbbe9e4cfe0d0008d2a4c9
😎 Deploy Preview https://deploy-preview-1447.kargo.akuity.io
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@mihaigalos mihaigalos mentioned this pull request Feb 1, 2024
Closed
3 tasks
@mihaigalos
Copy link
Author

Please help with setup of the toolchain on Apple M1. Seems I could install protobuf and the buf command runs successfully, however the rest fails.

Commandline output - click to expand! 
make codegen
buf generate api
controller-gen \
                rbac:roleName=manager-role \
                crd \
                webhook \
                paths=./api/... \
                output:crd:artifacts:config=charts/kargo/crds
F0201 12:42:02.040296   84008 packages.go:59] Failed loading boilerplate: open k8s.io/kubernetes/hack/boilerplate/boilerplate.go.txt: no such file or directory

goroutine 1 [running]:
github.com/golang/glog.Fatalf(...)
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog.go:556
github.com/llparse/controller-gen/generators.Packages(0x14000158700, 0x14000138580)
        /Users/mihai.galos/go/pkg/mod/github.com/llparse/controller-gen@v0.0.0-20180131011002-7a38c4658cb4/generators/packages.go:59 +0xb0
k8s.io/gengo/args.(*GeneratorArgs).Execute(0x14000138580, 0x0?, {0x1047998e1, 0x6}, 0x1048d53a0)
        /Users/mihai.galos/go/pkg/mod/k8s.io/gengo@v0.0.0-20230829151522-9cce18d56c01/args/args.go:193 +0x188
main.main()
        /Users/mihai.galos/go/pkg/mod/github.com/llparse/controller-gen@v0.0.0-20180131011002-7a38c4658cb4/main.go:39 +0x344

SIGABRT: abort
PC=0x188b11af8 m=0 sigcode=0

goroutine 0 [idle]:
syscall.syscall(0x1048b8fa0?, 0x104b0b8a0?, 0x1400016a120?, 0x0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/sys_darwin.go:23 +0x58 fp=0x140001d1910 sp=0x140001d1880 pc=0x104518e98
syscall.kill(0x140001d1988?, 0x10?, 0x1048b8fa0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/syscall/zsyscall_darwin_arm64.go:367 +0x34 fp=0x140001d1940 sp=0x140001d1910 pc=0x104560e64
syscall.Kill(...)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/syscall/syscall_darwin.go:124
os.(*Process).signal(0x14828?, {0x1048d85e0?, 0x10482fe18?})
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/os/exec_unix.go:78 +0x10c fp=0x140001d19b0 sp=0x140001d1940 pc=0x10458610c
os.(*Process).Signal(...)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/os/exec.go:140
github.com/golang/glog.abortProcess()
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog_file_posix.go:40 +0x38 fp=0x140001d19e0 sp=0x140001d19b0 pc=0x10478ac68
github.com/golang/glog.fatalf(0x14000138580?, {0x1047c0747?, 0x14000161a88?}, {0x14000118370?, 0x14000161b28?, 0x1044bf6c8?})
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog.go:517 +0x60 fp=0x140001d1a40 sp=0x140001d19e0 pc=0x104788e20
github.com/golang/glog.Fatalf(...)
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog.go:556
github.com/llparse/controller-gen/generators.Packages(0x14000158700, 0x14000138580)
        /Users/mihai.galos/go/pkg/mod/github.com/llparse/controller-gen@v0.0.0-20180131011002-7a38c4658cb4/generators/packages.go:59 +0xb0 fp=0x140001d1db0 sp=0x140001d1a40 pc=0x104794600
k8s.io/gengo/args.(*GeneratorArgs).Execute(0x14000138580, 0x0?, {0x1047998e1, 0x6}, 0x1048d53a0)
        /Users/mihai.galos/go/pkg/mod/k8s.io/gengo@v0.0.0-20230829151522-9cce18d56c01/args/args.go:193 +0x188 fp=0x140001d1e10 sp=0x140001d1db0 pc=0x104771b08
main.main()
        /Users/mihai.galos/go/pkg/mod/github.com/llparse/controller-gen@v0.0.0-20180131011002-7a38c4658cb4/main.go:39 +0x344 fp=0x140001d1f30 sp=0x140001d1e10 pc=0x104797e64
runtime.main()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:267 +0x2bc fp=0x140001d1fd0 sp=0x140001d1f30 pc=0x1044ebdfc
runtime.goexit()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/asm_arm64.s:1197 +0x4 fp=0x140001d1fd0 sp=0x140001d1fd0 pc=0x10451c7e4

goroutine 1 [syscall]:
syscall.syscall(0x1048b8fa0?, 0x104b0b8a0?, 0x1400016a120?, 0x0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/sys_darwin.go:23 +0x58 fp=0x140001d1910 sp=0x140001d1880 pc=0x104518e98
syscall.kill(0x140001d1988?, 0x10?, 0x1048b8fa0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/syscall/zsyscall_darwin_arm64.go:367 +0x34 fp=0x140001d1940 sp=0x140001d1910 pc=0x104560e64
syscall.Kill(...)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/syscall/syscall_darwin.go:124
os.(*Process).signal(0x14828?, {0x1048d85e0?, 0x10482fe18?})
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/os/exec_unix.go:78 +0x10c fp=0x140001d19b0 sp=0x140001d1940 pc=0x10458610c
os.(*Process).Signal(...)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/os/exec.go:140
github.com/golang/glog.abortProcess()
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog_file_posix.go:40 +0x38 fp=0x140001d19e0 sp=0x140001d19b0 pc=0x10478ac68
github.com/golang/glog.fatalf(0x14000138580?, {0x1047c0747?, 0x14000161a88?}, {0x14000118370?, 0x14000161b28?, 0x1044bf6c8?})
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog.go:517 +0x60 fp=0x140001d1a40 sp=0x140001d19e0 pc=0x104788e20
github.com/golang/glog.Fatalf(...)
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog.go:556
github.com/llparse/controller-gen/generators.Packages(0x14000158700, 0x14000138580)
        /Users/mihai.galos/go/pkg/mod/github.com/llparse/controller-gen@v0.0.0-20180131011002-7a38c4658cb4/generators/packages.go:59 +0xb0 fp=0x140001d1db0 sp=0x140001d1a40 pc=0x104794600
k8s.io/gengo/args.(*GeneratorArgs).Execute(0x14000138580, 0x0?, {0x1047998e1, 0x6}, 0x1048d53a0)
        /Users/mihai.galos/go/pkg/mod/k8s.io/gengo@v0.0.0-20230829151522-9cce18d56c01/args/args.go:193 +0x188 fp=0x140001d1e10 sp=0x140001d1db0 pc=0x104771b08
main.main()
        /Users/mihai.galos/go/pkg/mod/github.com/llparse/controller-gen@v0.0.0-20180131011002-7a38c4658cb4/main.go:39 +0x344 fp=0x140001d1f30 sp=0x140001d1e10 pc=0x104797e64
runtime.main()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:267 +0x2bc fp=0x140001d1fd0 sp=0x140001d1f30 pc=0x1044ebdfc
runtime.goexit()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/asm_arm64.s:1197 +0x4 fp=0x140001d1fd0 sp=0x140001d1fd0 pc=0x10451c7e4

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:398 +0xc8 fp=0x1400005af90 sp=0x1400005af70 pc=0x1044ec258
runtime.goparkunlock(...)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:404
runtime.forcegchelper()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:322 +0xb8 fp=0x1400005afd0 sp=0x1400005af90 pc=0x1044ec0b8
runtime.goexit()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/asm_arm64.s:1197 +0x4 fp=0x1400005afd0 sp=0x1400005afd0 pc=0x10451c7e4
created by runtime.init.6 in goroutine 1
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:310 +0x24

goroutine 18 [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:398 +0xc8 fp=0x14000056760 sp=0x14000056740 pc=0x1044ec258
runtime.goparkunlock(...)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:404
runtime.bgsweep(0x0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mgcsweep.go:280 +0xa0 fp=0x140000567b0 sp=0x14000056760 pc=0x1044d72e0
runtime.gcenable.func1()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mgc.go:200 +0x28 fp=0x140000567d0 sp=0x140000567b0 pc=0x1044cbdb8
runtime.goexit()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/asm_arm64.s:1197 +0x4 fp=0x140000567d0 sp=0x140000567d0 pc=0x10451c7e4
created by runtime.gcenable in goroutine 1
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mgc.go:200 +0x6c

goroutine 19 [GC scavenge wait]:
runtime.gopark(0x1400009e000?, 0x10482fd98?, 0x1?, 0x0?, 0x140000924e0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:398 +0xc8 fp=0x14000056f50 sp=0x14000056f30 pc=0x1044ec258
runtime.goparkunlock(...)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:404
runtime.(*scavengerState).park(0x104b0aca0)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mgcscavenge.go:425 +0x5c fp=0x14000056f80 sp=0x14000056f50 pc=0x1044d4b5c
runtime.bgscavenge(0x0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mgcscavenge.go:653 +0x44 fp=0x14000056fb0 sp=0x14000056f80 pc=0x1044d50b4
runtime.gcenable.func2()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mgc.go:201 +0x28 fp=0x14000056fd0 sp=0x14000056fb0 pc=0x1044cbd58
runtime.goexit()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/asm_arm64.s:1197 +0x4 fp=0x14000056fd0 sp=0x14000056fd0 pc=0x10451c7e4
created by runtime.gcenable in goroutine 1
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mgc.go:201 +0xac

goroutine 34 [finalizer wait]:
runtime.gopark(0x140001041a0?, 0x32?, 0x98?, 0x1?, 0x1048d0fe0?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:398 +0xc8 fp=0x1400005a580 sp=0x1400005a560 pc=0x1044ec258
runtime.runfinq()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mfinal.go:193 +0x108 fp=0x1400005a7d0 sp=0x1400005a580 pc=0x1044caea8
runtime.goexit()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/asm_arm64.s:1197 +0x4 fp=0x1400005a7d0 sp=0x1400005a7d0 pc=0x10451c7e4
created by runtime.createfing in goroutine 1
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/mfinal.go:163 +0x80

goroutine 35 [select]:
runtime.gopark(0x140000b0f80?, 0x2?, 0x58?, 0xe?, 0x140000b0f64?)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/proc.go:398 +0xc8 fp=0x140000b0e10 sp=0x140000b0df0 pc=0x1044ec258
runtime.selectgo(0x140000b0f80, 0x140000b0f60, 0x0?, 0x0, 0x0?, 0x1)
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/select.go:327 +0x608 fp=0x140000b0f20 sp=0x140000b0e10 pc=0x1044fcb38
github.com/golang/glog.(*fileSink).flushDaemon(0x104b0aa98)
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog_file.go:351 +0xa8 fp=0x140000b0fb0 sp=0x140000b0f20 pc=0x10478a908
github.com/golang/glog.init.1.func1()
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog_file.go:166 +0x28 fp=0x140000b0fd0 sp=0x140000b0fb0 pc=0x104789ae8
runtime.goexit()
        /Users/mihai.galos/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.3.darwin-arm64/src/runtime/asm_arm64.s:1197 +0x4 fp=0x140000b0fd0 sp=0x140000b0fd0 pc=0x10451c7e4
created by github.com/golang/glog.init.1 in goroutine 1
        /Users/mihai.galos/go/pkg/mod/github.com/golang/glog@v1.1.2/glog_file.go:166 +0x150

r0      0x0
r1      0x0
r2      0x1
r3      0x104b0b040
r4      0x7c0
r5      0x14000002230
r6      0x1
r7      0x0
r8      0x104b0b8a0
r9      0x140001d1898
r10     0x0
r11     0x0
r12     0x1045635e0
r13     0x16b94edf0
r14     0x7
r15     0x302e302e3076406e
r16     0x25
r17     0x188b11ae8
r18     0x0
r19     0x140001d10f8
r20     0x14000145810
r21     0x14000145860
r22     0x104799c4d
r23     0x16b94ef00
r24     0x16b94ef40
r25     0x1047986b1
r26     0x0
r27     0x810
r28     0x104b0b040
r29     0x16b94edd8
lr      0x10451ddcc
sp      0x16b94edd0
pc      0x188b11af8
fault   0x188b11af8
make: *** [codegen] Error 2

@mihaigalos
feat(cli): Support for separate write repo
9e9b5f4 
Signed-off-by: Mihai Galos <mihai.galos@rbinternational.com>
@mihaigalos
feat(cli): Support for separate branch
4c56116 
Signed-off-by: Mihai Galos <mihai.galos@rbinternational.com>
mihaigalos
mihaigalos commented Feb 1, 2024
View reviewed changes
api/v1alpha1/stage_types.go
//
//+kubebuilder:validation:MinLength=1
//+kubebuilder:validation:Pattern=`^https://(\w+([\.-]\w+)*@)?\w+([\.-]\w+)*(:[\d]+)?(/.*)?$`
WriteRepoURL string `json:"writeRepoURL"`
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment on setting up the toolchain. I think this should generate a new types.pb.go and the Getters/Setters for this field.

@mihaigalos
chore: Remove superfluous
08feb00 
Signed-off-by: Mihai Galos <mihai.galos@rbinternational.com>
@krancour
Copy link
Member

krancour commented Feb 1, 2024

@mihaigalos thank you for the draft.

Easy part first...

Please help with setup of the toolchain on Apple M1.

https://kargo.akuity.io/contributor-guide/hacking-on-kargo#executing-code-generation

The harder part:

There wasn't any commitment to #1354 from maintainers yet. I am sure that at an initial glance, this looks like trivial change, but unfortunately, it is not.

Two very big considerations:

  1. This isn't going to work for Kargo Render (which was actually the ask in Support of rendering to another repository #1354. When Kargo invokes Kargo Render (which is a separate tool, that can be used standalone), there is no provision for instructing it to read from one repo, but write to another. And while I believe Kargo Render could tentatively add that feature, there would need to be some discussion among the maintainers as to whether it is contrary to what Kargo Render is trying to achieve. I noted this in Support of rendering to another repository #1354:

    One of the fundamental premises of Kargo Render is that Kargo Render configuration is "close" to the source it operates on and to the branches it renders. i.e. If you were looking at repo, you would see what got rendered, what it got rendered from, and what the rules were for doing that, all in one place.

    While this generally seems doable, it upsets the apple cart a little bit in terms of that fundamental premise of Kargo Render.

  2. Where Kargo writes things is an important consideration for health checks that Kargo performs. Typically, Stages are backed by one or more Argo CD Applications and Kargo monitors those to ensure they are healthy and synced to the correct revision. Those are factored into determining overall Stage health.

    In cases where Kargo has written to a different branch (which is typical), it records a "health check commit" in addition to information about the commit from the trunk that the Freight references. Naturally, these will be different. Kargo knows, when performing a health check, that if the health check commit field is non-empty, then that is the commit the Application should be synced to if it is to be judged healthy.

    I haven't fully thought through the consequences yet of the health check commit being in a different repo than the commit that the Freight references. Things might ok, because I don't believe the health checks currently check what repos the Apps are synced to; they only check the commit. (Honestly, they ought to check what repo the Apps are synced to.) Even if this will technically work, there is some information loss if, in addition to recording a health check commit, we're not also recording a health check repo. i.e. This commit in this repo is where the correct config for this Stage can be found.

I don't want any of this to sound discouraging. No one is saying no to #1354 (at least not yet). We are, however, obsessive about user experience, so what really needs to happen first is a more thorough discussion around the impact of this change and the proposed implementation.

I do not want to scrap this, but I propose putting a pin in it until more discussion has happened in #1354. We're working hard to get v0.4.0 out right now, so I will commit to after that, getting an earnest discussion going about this.

@krancour
Copy link
Member

#2537 has this covered.

@krancour krancour closed this Sep 26, 2024
@mihaigalos mihaigalos deleted the feat/repo/write branch September 27, 2024 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mihaigalos @krancour