You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently Catalogd uses a Pod for unpacking catalog image contents and reads the unpacked contents from said Pod’s logs. There are a few problems associated with this:
As mentioned by the Kubernetes documentation on working with Pods, Pods are designed as disposable entities. Pods that have exited successfully can be garbage collected at any time and thus there is no guarantee that it will still exist by the time we attempt to read the logs or that the logs that are present are lossless.
It is not recommended to manage Pod resources yourself and instead use something like a Deployment, Job, or StatefulSet.
When thinking about polling, we will likely be checking if the SHA of an image is different from the last time the image was unpacked. The general approach to doing this with a Pod based unpacking process is to create another pod to pull the same image and check the Pod status to determine if there is a difference in the SHA.
Directly communicating with an image registry allows us to avoid running into these issues altogether.
There are several issues related to our use of a pod to unpack images:
https://github.com/operator-framework/catalogd/issues/22
https://github.com/operator-framework/catalogd/issues/66
https://github.com/operator-framework/catalogd/issues/132
I propose that we switch to an image registry client used directly by the controller. I think for the initial implementation we can:
Always resolve tags to SHAs by communicating with image registry
Pull content for tag into an emptyDir volume
i. If the content for a SHA is cached locally, no need to pull again
Unpack content into served filesystem
i. If the desired SHA is already being served, no need to unpack again
Future optimizations could include:
Running full resolve/unpack process in a goroutine, and using a channel to communicate events back to the reconciler to requeue
Rate-limit requests to an image registry (but if a request is rate-limited always put another request in the queue to make sure we don't rest on a stale tag resolution.
Explore cache optimizations (e.g. persistent volume, garbage collection based on LRU + max inactive size)
Overview
Currently Catalogd uses a Pod for unpacking catalog image contents and reads the unpacked contents from said Pod’s logs. There are a few problems associated with this:
Directly communicating with an image registry allows us to avoid running into these issues altogether.
Related Issues
Design Documents
RFC: https://docs.google.com/document/d/1VHlUAVewIckVMLccNsoNptRsrQ19KcWzQ2HZetYK2h0/edit?usp=sharing
Task List
Unpacker
implementation #145Unpack
implementation #182The text was updated successfully, but these errors were encountered: