Package wait is too late

[blancharda]

Overview

The common/zarf.yaml package definition defines a wait action to ensure the uds package object has been properly handled by pepr. However, the action runs after the gitlab application chart is applied, which in rare edge cases could create a race.

I propose we split the charts into separate components to ensure the uds-package has been handled and is Ready prior to applying the gitlab chart.

[mjnagel]

Would love to better understand the scenarios in which this is problematic. I am tracking a few things that are "dependencies" of the GitLab deployment (and more broadly of any chart deployments using Package CRs):

• Network Policies: If these don't exist, the components of GitLab wouldn't be able to communicate (and similarly wouldn't be able to hit external services). In general my experience has been that services are resilient and will retry any connections so an intermittent network policy blocking a request shouldn't fail the deployment?
• SSO Secret: One or more of the GitLab pods mount this secret, but k8s scheduling should prevent the pod from starting up until it is present.

Is there a specific example you could give of a problem caused by this timing? Just want to better understand what specific problem is arising here. We have definitely discussed a few different paths forward here:

• Splitting the components - in this case I think it might make a lot of sense. It would technically be a breaking change I suspect because of bundle overrides needing to be different.
• UDS-CLI natively having CRD knowledge of Packages and waiting on them
• Zarf providing per chart actions (maybe furthest from our control)

Edit: Should also note additional details around network policies...pepr will apply them first in the flow of the Package reconciliation. After that it will istio-inject the namespace and cycle any pods that pre-existed injection (ensuring that no pre-netpol connections can be maintained).