add rest api / webhook implementation to trigger warehouse/freight refresh

[lknite]

Checklist

• I've searched the issue queue to verify this is not a duplicate feature request.

Proposed Feature

Implement a REST API / webhook which takes the freight repo/name (exactly as defined in the warehouse as a subscription) & causes a refresh of all warehouses in all projects polling that freight (or at least all warehouses with an 'annotation' indicating they want to be included, but wouldn't all warehouses in all projects want to be included?).

If this feature already exists, then maybe we could add a note to the getting started documentation.

Motivation

I love the idea of dedicating pipelines to building an image or image & helm chart ... and nothing else, with promotion all handled via another application, kargo.

However, my developers are not going to want to wait for kargo to poll an image or helm repository, they need the next step to be kicked off immediately.

Until kargo can handle webhooks from repos indicating there is new freight available, this could be a workaround everyone uses.

Workaround

Until a preferred solution comes along, I'm implementing something quick in my environment to delete warehouse resources. The warehouses are then reapplied automatically via argocd and immediately refresh freight.

Suggested Implementation

• implement the ability to generate an apiKey
• implement a restapi which takes the apiKey, subscription details, and performs the necessary refresh
• add an annotation to warehouses which enable refreshes via the api (though, wouldn't they all want this by default?)
  • kargo.akuity.io/webhook-enabled: true
• add an annotation on the warehouse to authorize in lieu of an apiKey:
  • kargo.akuity.io/webhook-anonymous-allowed: true
  • kargo.akuity.io/webhook-authorized: gitlab.com,10.0.0.0/24

[lknite]

Until something is available, here's a workaround folks can use in the meantime in your pipelines:

APISERVER="https://10.1.2.3:6443"
curl -X DELETE $APISERVER/apis/kargo.akuity.io/v1alpha1/namespaces/<kargo-project>/warehouses --header "Authorization: Bearer $TOKEN" --insecure

Get the token via a service account assigned the clusterrole to delete a warehouse:

$ cat clusterrole-webhook.yaml 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kargo-webhook
rules:
- apiGroups: ["kargo.akuity.io"]
  #apiGroups: ["kargo.akuity.io/v1alpha1"]
  resources: ["warehouses"]
  verbs: ["get", "list", "delete", "deletecollection"]
  
$ cat clusterrolebinding-webhook.yaml 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kargo-webhook
subjects:
- kind: ServiceAccount
  name: kargo-webhook
  namespace: kargo
roleRef:
  kind: ClusterRole
  name: kargo-webhook
  apiGroup: rbac.authorization.k8s.io

$ cat serviceaccount-webhook.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: null
  name: kargo-webhook

$ cat serviceaccounttoken-webhook.yaml 
apiVersion: v1
kind: Secret
metadata:
  name: kargo-webhook
  annotations:
    kubernetes.io/service-account.name: kargo-webhook
type: kubernetes.io/service-account-token

[krancour]

I'm going to close this because we already have #647 and #1449 tracking this from both a git repo and container repo perspective. Note that a universal solution is rather difficult, but we intend to do this eventually.

In the meantime, there is a simpler, less heavy-handed workaround than the one you're using currently. Instead of deleting a Warehouse, forcing Argo CD to re-create it and thereby forcing a reconciliation, consider adding the annotation kargo.akuity.io/refresh with the value being an RFC3339 timestamp. This isn't documented because it's not really part of any public API, but it is how the Kargo CLI and UI trigger a refresh...

So, as an even simpler workaround, you could actually use the CLI to do this:

kargo refresh warehouse foo --project bar

[lknite]

Sounds good. Thanks for the refresh annotation. I'll update my technique.

[lknite]

For reference:

Script to trigger refresh:

$ cat refresh-kargo-project.sh 
#!/bin/sh

APISERVER=https://1.2.3.4:6443
APITOKEN=$(cat ./token)
NAMESPACE=kargo-project-namespace
WAREHOUSE=warehouse

ATTR_NAME=kargo.akuity.io/refresh
ATTR_VALUE=$(date --rfc-3339=seconds)

curl -sSk \
    -X PATCH \
    -d "{\"metadata\": {\"annotations\": {\"${ATTR_NAME}\": \"${ATTR_VALUE}\"} } } " \
    -H "Authorization: Bearer ${APITOKEN}" \
    -H 'Content-Type: application/merge-patch+json' \
    ${APISERVER}/apis/kargo.akuity.io/v1alpha1/namespaces/${NAMESPACE}/warehouses/${WAREHOUSE}

RBAC / ServiceAccount related:

$ cat serviceaccount-gitlab.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: null
  name: gitlab

$ cat clusterrole-gitlab.yaml 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: warehouse-refresh
rules:
- apiGroups: ["kargo.akuity.io"]
  resources: ["warehouses"]
  verbs: ["patch"]
  
$ cat clusterrolebinding-gitlab.yaml 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: warehouse-refresh
subjects:
- kind: ServiceAccount
  name: gitlab
  namespace: kargo
roleRef:
  kind: ClusterRole
  name: warehouse-refresh
  apiGroup: rbac.authorization.k8s.io
  
$ cat secret-serviceaccount-gitlab.yaml 
apiVersion: v1
kind: Secret
metadata:
  name: serviceaccount-gitlab
  annotations:
    kubernetes.io/service-account.name: gitlab
type: kubernetes.io/service-account-token