Consider allowing base image selection based on platform

[tcnghia]

Currently I wasn't able to find a base image that has both Windows and Linux. I am considering stitching the official Windows image with distroless to have such a base image. However, a user going down that path will need to maintain this stitching themselves.

Would it be bad for ko to allow selecting base image based on platform? Then with a couple more lines in .ko.yaml, ko publish --platform=all will easily 3x its OS x ARCH combination.

[imjasonh]

I am considering stitching the official Windows image with distroless to have such a base image.

This is exactly what Tekton plans to do, stitching together distroless+Windows with https://github.com/tektoncd/plumbing/tree/main/cmd/combine

I'd be willing to entertain ideas about how ko could support this better without other tools, if you have any ideas I'm all ears. 👂

[tcnghia]

I don't know how ko's magic work underneath, but let me try describing the UX. Please tell me if that isn't possible or too difficult.

How about in ko.yaml:

defaultBaseImage: distroless, windows/base/image, other/base/image

When a variant is provided by more than one base image we will only consider the first one.

[jonjohnsonjr]

X-ref google/go-containerregistry#1144

[imjasonh]

I played around with this a bit at main...imjasonh:combine-base-images

Basically, allowing defaultBaseImage to contain commas, splitting on those commas, and combining the indexes specified by those refs, copying cmd/combine code.

$ crane manifest $(go run ./ build --platform=all ./test/) | jq
2021/10/12 15:06:17 Using base gcr.io/distroless/static:nonroot for github.com/google/ko/test
2021/10/12 15:06:19 Using base mcr.microsoft.com/windows/nanoserver:1809 for github.com/google/ko/test
2021/10/12 15:06:20 Building github.com/google/ko/test for linux/amd64
2021/10/12 15:06:21 Building github.com/google/ko/test for linux/arm
2021/10/12 15:06:21 Building github.com/google/ko/test for linux/arm64
2021/10/12 15:06:22 Building github.com/google/ko/test for linux/ppc64le
2021/10/12 15:06:23 Building github.com/google/ko/test for linux/s390x
2021/10/12 15:06:24 Building github.com/google/ko/test for windows/amd64
...
2021/10/12 15:06:39 Published gcr.io/imjasonh/test-46c4b272b3716c422d5ff6dfc7547fa9@sha256:b81fcbfe76462f66a2aecde2bdaabe89dc34b15aa1018975b11a9f8219938811

Manifest JSON

``` { "schemaVersion": 2, "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", "manifests": [ { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 752, "digest": "sha256:657997de10efe5a246843ecdf425fe8c1046ee2bf78e8f55658d7e393f8c04b4", "platform": { "architecture": "amd64", "os": "linux" } }, { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 752, "digest": "sha256:7ea5ce75ebdf3116a92cc8977e351a9f8b2045c4985be7f82c122b66cb12ae66", "platform": { "architecture": "arm", "os": "linux" } }, { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 752, "digest": "sha256:f78a9c33ce8aaa25bdecac1433e07cdfb6da2a15b6bcf19232388d114456947a", "platform": { "architecture": "arm64", "os": "linux" } }, { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 752, "digest": "sha256:7818321e8eefeec9287aca8f79ed02105bb93fe697b8901544d3ce4e61b3e85f", "platform": { "architecture": "ppc64le", "os": "linux" } }, { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 752, "digest": "sha256:fe3cc4ca545b467b12e960cb9d5529c7209463bf032b61cac703ed34e282337d", "platform": { "architecture": "s390x", "os": "linux" } }, { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 898, "digest": "sha256:6dc63b502c86e16f033ba03f9d815e1744ee0396c77d93a3458a42ad66038956", "platform": { "architecture": "amd64", "os": "windows", "os.version": "10.0.17763.2237" } } ] } ```

One thing that breaks is setting base image annotations (#221) since there are now multiple bases, or depending on how you think about it, the base is an ephemeral index that doesn't exist except during build-time. I'm not really sure how to address that, or if this points to a fundamental issue with doing this this way.

[tcnghia]

I played around with this a bit at main...imjasonh:combine-base-images

This is very awesome. Thanks a lot for hacking this!

One thing that breaks is setting base image annotations (#221) since there are now multiple bases, or depending on how you think about it, the base is an ephemeral index that doesn't exist except during build-time. I'm not really sure how to address that, or if this points to a fundamental issue with doing this this way.

I see, in cmd/combine we don't need to have a base image for the combined image, since the combined image is the base image. In ko, where we do want to set the base image, we can't.

Can we push two images:

• one untagged for the combined base
• one tagged image contains koapp on top of the combined base

However if we do this we are just simulating cmd/combine and ko in a single command... I think it is cleaner if we use both cmd/combine (or a crane equivalence) and ko. If the users always run one after the other, they don't need to maintain anything separately.

[tcnghia]

I don't have anything to add here, beside that I now would prefer the approach of building the base image separately now. I'm closing the issue. Please feel free to reopen if you have something in mind. Thanks a lot for prototyping and taking the time to explain!