Artifact registry support

[SteveLasker]

Additions to enhance distribution for supporting additional artifact types, such as Helm Charts, Singularity Images for High Performance Computing workloads, Open Policy Agent Bundles

[jzelinskie]

I'll reiterate what I've said in every meeting on this topic so far: even though I strongly believe in this use case, I'm not sure that a specification for artifact registries should live in the Open Containers Initiative. However, in the meantime this still might be a good place to work on the idea.

I'm also still not convinced that we've come to a consensus on a cohesive strategy for reusing the existing distribution spec in a way that satisfies everyone. From our previous discussion, it seemed like we still require alignment from Docker's side of CNAB to have confidence in a model that everyone can properly leverage.

[glyn]

It would be good to explain why it is desirable to store non-image artefacts in a registry. Does this only really make sense for artefacts which reference images?

[SteveLasker]

As a spec, I struggled with how much "pitch" to put into the spec, as opposed to speaking to the details. I've written a few different posts, done presentations to set the overall context:

• Blog Post: Cloud Native Artifact Registries evolve from Docker Container Registries
• PowerPoint: Leveraging OCI Distribution for new Cloud Native Artifacts

Do you think we should put more of the pitch here? Are you looking for what type of artifacts to store?

[glyn]

As a spec, I struggled with how much "pitch" to put into the spec, as opposed to speaking to the details. I've written a few different posts, done presentations to set the overall context:

• Blog Post: Cloud Native Artifact Registries evolve from Docker Container Registries
• PowerPoint: Leveraging OCI Distribution for new Cloud Native Artifacts

Thanks but I don't think it would be appropriate to reference these in the spec.

Do you think we should put more of the pitch here?

Yes, but not framed as a sales pitch. I'm looking for rationale of when it makes sense to use this feature.

Are you looking for what type of artifacts to store?

Yes. In particular, does it make any sense to store an artefact that does not reference images?

[SteveLasker]

Agree we shouldn't reference these within the spec. I was more providing context on how I was framing things, outside the spec. To provide a funnel of the scenarios, to the specifics.

As for when and what artifact types, I don't know if we should attempt to limit. The question I ask in the above Cloud Native Rejekts talk is; why would you create Yet Another Storage Solution (YASS).

Registries have become ubiquitous. It's hard to see modern workloads that don't use containers. Even IoT devices and cars are running containers because of their packaging, isolation, patching, testing and deployment models. But, lets assume you have an optimized environment that doesn't require container images. Could you justify the engineering effort to build YASS? Or, would you just implement one of the current distribution solutions, and store the artifact types in it?

[jdolitsky]

Can somebody from Docker/DockerHub please weigh in on this approach and any potential shortcomings? Quay? ECR? GCR?

Would love to see the hubs come together on a standard approach for mulitartifact support, whether it is how Steve has proposed, or some alternation.

Here is probably the most significant piece, which is on how registries reason over artifact types on the manifest level:

Artifacts are defined by setting the manifest.config.mediaType to a globally unique value. The following format is used to differentiate the type of artifact:

application/vnd.[org|company].[objectType].[optionalSubType].config.[version]+json

The rest of the changes are about new file artifactMapping.json, which is just a format projects can use to establish their custom artifact type (that registries can then support "officially").

[dmcgowan]

I'll sum up my recommendation from the OCI call earlier

• Remove specific mention of artifact types. Propose a new repository under opencontainers Github org to the OCI TOB which defines the artifact use case and known examples of artifacts used by OCI manifests. The first example of this is the OCI image with clarification about the history which lead the OCI image type and the OCI manifest (a generic envelope type) to live within the same specification.
• Remove wording within the distribution specification which mentions the distribution spec as used for distributing images. We can use the word artifact there as this PR does, just don't want that word overused. I have a slight preference for using the word manifest/manifest-index generically in this spec to refer to the objects stored by registries, but I understand that may be confusing.
• Replace the parts of this PR which update the spec to discuss artifacts with links to the above artifact repository once it is created. That repository can be a place for these discussions while allowing the distribution spec to stay generic and versioned separately from the types that use it.

[caniszczyk]

re: new project for @opencontainers/tob, have you decided what you will call this and who will put forth the proposal? happy to work with them

[mikebrow]

I'll sum up my recommendation from the OCI call earlier

• Remove specific mention of artifact types. Propose a new repository under opencontainers Github org to the OCI TOB which defines the artifact use case and known examples of artifacts used by OCI manifests. The first example of this is the OCI image with clarification about the history which lead the OCI image type and the OCI manifest (a generic envelope type) to live within the same specification.
• Remove wording within the distribution specification which mentions the distribution spec as used for distributing images. We can use the word artifact there as this PR does, just don't want that word overused. I have a slight preference for using the word manifest/manifest-index generically in this spec to refer to the objects stored by registries, but I understand that may be confusing.
• Replace the parts of this PR which update the spec to discuss artifacts with links to the above artifact repository once it is created. That repository can be a place for these discussions while allowing the distribution spec to stay generic and versioned separately from the types that use it.

SGTM

[SteveLasker]

Hi folks, I’m out for a few days of vacation, but will be back Monday for follow-up to the call and all the feedback from Derek, Mike and others. Steve

[SteveLasker]

I've create the new TOB issue here: opencontainers/tob#59
If we can agree to the repo, I can start on the content.

[vbatts]

I think this effort has sufficiently become its own thing. Any references can be done in a separate PR now.