Add command pack manifest create to create a manifest list

[jjbustamante]

Context

We've been working on the idea to enable multi-arch support on CNB. A mentorship project was created on LFX and this feature is part of the efforts to build the Minimal Viable Product (MVP) during the program duration.

More information can be found:

• Notes
• Meeting recording

Description

Currently, the samples repository contains buildpacks and meta-buildpacks examples, using an ER diagram, we can represent those buildpacks as follow:

erDiagram
    HELLO_WORLD_BP ||--o{ HELLO_UNIVERSE_MBP : contains
    HELLO_WORLD_BP{
        string plaform_os
        string plaform_arch
    }

    HELLO_MOON_BP ||--o{ HELLO_UNIVERSE_MBP : contains
    HELLO_MOON_BP{
        string plaform_os
        string plaform_arch
    }
  
    HELLO_WORLD_WINDOWS_BP ||--o{ HELLO_UNIVERSE_WINDOWS_MBP : contains
    HELLO_WORLD_WINDOWS_BP {
        string plaform_os
        string plaform_arch
    }

    HELLO_MOON__WINDOWS_BP ||--o{ HELLO_UNIVERSE_WINDOWS_MBP : contains
    HELLO_MOON__WINDOWS_BP{
        string plaform_os
        string plaform_arch
    }

    HELLO_UNIVERSE_MBP {
        string plaform_os
        string plaform_arch
    }
    HELLO_UNIVERSE_WINDOWS_MBP {
        string plaform_os
        string plaform_arch
    }

Loading

These buildpacks are hosted in our docker hub repository and are available for Linux and Windows OS.

The idea is to create a new command pack buildpack manifest create to create a manifest list and combine those buildpacks. An update version of our previous ER-diagram will be:

erDiagram
    HELLO_WORLD_BP ||--o{ HELLO_UNIVERSE_MBP : contains
    HELLO_WORLD_BP{
        string plaform_os
        string plaform_arch
    }

    HELLO_MOON_BP ||--o{ HELLO_UNIVERSE_MBP : contains
    HELLO_MOON_BP{
        string plaform_os
        string plaform_arch
    }
  
    HELLO_WORLD_WINDOWS_BP ||--o{ HELLO_UNIVERSE_WINDOWS_MBP : contains
    HELLO_WORLD_WINDOWS_BP {
        string plaform_os
        string plaform_arch
    }

    HELLO_MOON__WINDOWS_BP ||--o{ HELLO_UNIVERSE_WINDOWS_MBP : contains
    HELLO_MOON__WINDOWS_BP{
        string plaform_os
        string plaform_arch
    }

    HELLO_UNIVERSE_MBP {
        string plaform_os
        string plaform_arch
    }
    HELLO_UNIVERSE_WINDOWS_MBP {
        string plaform_os
        string plaform_arch
    }

    HELLO_UNIVERSE_WINDOWS_MBP ||--o{ HELLO_MULTIARCH_UNIVERSE : "contains manifest"
    HELLO_UNIVERSE_MBP ||--o{ HELLO_MULTIARCH_UNIVERSE : "contains manifest"


    HELLO_MULTIARCH_UNIVERSE {
        manifest[] manifests
        string mediaType
    }

Loading

Proposed solution

A new feature on Pack can be implemented to help our users. The feature is inspired by the similar functionality in docker or podman

pack manifest create <manifest-list> <manifest> [<manifest> ... ]
  options:
       --format   Manifest list type (oci or v2s2) to use when pushing the list (default is v2s2)
       --publish Push a manifest list to a repository

Validations:

• When the publish flag is used all the manifests MUST have os/arch defined otherwise an error must be thrown
• The format flag will be ignored if it is not used in conjunction with the publish flag

Example

Based on the previous ER-diagram, we want to create a cnbs/sample-package:hello-multiarch-universe that combines the Linux and the Windows version of our current buildpacks.

pack manifest create cnbs/sample-package:hello-multiarch-universe \ 
                                         cnbs/sample-package:hello-universe \ 
                                         cnbs/sample-package:hello-universe-windows

Using docker Media-Types:

By default the command will create a manifest list in the local storage using the docker media types (Version 2 schema 2 v2s2) with a content similar to:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 226083,
         "digest": "sha256: 87a832fd6a8d6995d336c740eb6f3da015401a6e564fcbe95ee1bf37557a8225",
         "platform": {
            "os": "linux",
           "architecture": "",
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 226083,
         "digest": "sha256:670d62fbee841d256a706801a03be9c84d37fc2cd6ef7538a7af9985c3d2ed8b",
         "platform": {
            "os": "windows",
           "architecture": ""
         }
      }   
   ]
}

Using OCI Media Types:

pack manifest create  --publish --format oci cnbs/sample-package:hello-multiarch-universe \ 
                                         cnbs/sample-package:hello-universe \ 
                                         cnbs/sample-package:hello-universe-windows 

Assuming all the manifests have defined os/arch the expected output is a manifest list created in a remote registry with a content similar to:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 226083,
         "digest": "sha256: 87a832fd6a8d6995d336c740eb6f3da015401a6e564fcbe95ee1bf37557a8225",
         "platform": {
            "os": "linux",
           "architecture": "amd64"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 226083,
         "digest": "sha256:670d62fbee841d256a706801a03be9c84d37fc2cd6ef7538a7af9985c3d2ed8b",
         "platform": {
            "os": "windows",
           "architecture": "amd64"
         }
      }   
   ]
}

Describe alternatives you've considered

Additional context

• This feature should be documented somewhere

[jjbustamante]

Husni @drac98 It is not ready, but let's start here.