feat: Implement Image Rescanning using Harbor Webhook API

[jopemachine]

Resolves #1913 (BA-143).

Important

Currently, the Image Rescanning through Webhook API implemented in this PR is only available for the HarborV2 registry.

Details

• Since the Harbor Webhook's request header cannot be customized, it was implemented in a way that bypasses the auth_middleware. It would be great to generalize this approach or find a better solution in the following works.

Implementation details

Events to Monitor

We will need to pay attention to the following events to ensure the image table is updated automatically.

• PUSH_ARTIFACT
  • Only the corresponding image is updated when an image artifact is created, copied, or updated.
• DELETE_ARTIFACT
  • It seems there is no feasible way to implement this. See feat: Implement Image Rescanning using Harbor Webhook API #3116 (comment).

Events to ignore

We won't need to care about the following events in this PR.

• PULL_ARTIFACT
• UPLOAD_CHART
• DOWNLOAD_CHART
• DELETE_CHART
• SCANNING_COMPLETED
• SCANNING_FAILED
• QUOTA_EXCEED
• QUOTA_WARNING
• REPLICATION

How to setup webhook

1. Configure registry webhook referring to Harbor webhook API documentation.

2. Add auth header configured in step 1 as webhook_auth_header to the ContainerRegistry row's extra column. Event handler is executed only when the webhook_auth_header equals the auth header value set in the webhook policy. If the auth header in the webhook policy is empty, the event handler will still be executed even if the webhook_auth_header is not set.

Tip

We don't need to subscribe to events for which event handlers are not implemented.

---

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue

---

📚 Documentation preview 📚: https://sorna--3116.org.readthedocs.build/en/3116/

---

📚 Documentation preview 📚: https://sorna-ko--3116.org.readthedocs.build/ko/3116/

[jopemachine]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: Implement Image Rescanning using Harbor Webhook API #3116 👈 (View in Graphite)
• feat: Implement CRUD API for managing Harbor per-project Quota #3090
• feat: Implement ImageNode, ImageNodes as per-project images API #3036
• feat: Implement Image RBAC project scope #3035
• feat: Implement APIs for associating, disassociating container_registries with groups #3067
• feat: Implement m2m table for container_registries, and groups #3065
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[jopemachine]

I tried to implement logic to remove the ImageRow upon receiving a DELETE_ARTIFACT event, but the webhook event does not include the architecture information of the image.

In the case of the PUSH_ARTIFACT event, the issue doesn't matter because the architecture information can be scanned via the Harbor artifact API before adding the ImageRow.

However, for DELETE_ARTIFACT, the event occurs after the image has already been deleted, so there is no way to retrieve the architecture information.

[jopemachine]

It would be ideal to check using registry_name, but the Harbor webhook request does not include registry_name information.

[HyeockJinKim]

I see a lot of other code accessing the RootContext field stored in request.app[“_root.context”], is there a reason why you stored it directly in the app here?
I'd like to know what separates the cases that access app directly from those that access RootContext. @achimnol

[jopemachine]

RootContext contains mutable configurations such as local_config and shared_config, as well as many objects that are frequently accessed throughout the codebase.

So I thought it might be better to inject hardcoded values into the app instead.
(But this is just my personal opinion.)

[HyeockJinKim]

This would be fine if the API was only used for harbor webhooks.

[jopemachine]

This would be fine if the API was only used for harbor webhooks.

The API currently supports only Harbor webhooks, but it could be expanded to accommodate other container registries's webhook in the future.

[HyeockJinKim]

I think this can be merged.