Provide a way to specify desired platform when creating packages and builders

[dmikusa]

Description

With the growing need for ARM64 images, it is a challenge to build them with the current tool support. It would be helpful if we had the ability to cross package buildpacks.

If I'm on an x86_64 system, I can cross-compile my buildpack code with Go (or I could be using something that doesn't need to be compiled like Bash), however, when I run pack buildpack package, it will only package the image for the architecture on which I'm presently running. That doesn't seem on the surface that it would be a technical requirement to assemble the image though, the architecture-specific files are being passed into the tool.

Proposed solution

I could see this being done in a few different ways:

1. As a flag to the pack cli.
2. As an option in package.toml, one could then specify a different package.toml file for each desired image
3. Both 1 and 2.

Describe alternatives you've considered

You can not do this, but it complicates the process as you then need a host OS for every os/architecture that you'd like to support.

Additional context

• This feature should be documented somewhere

[shruti1421]

Hey @dmikusa, I'm really interested to work on this project under LFX Mentorship 2023. Kindly guide me how to apply to this project and what are the resources required for solving this issue.

[dmikusa]

If you reach out on the project Slack -> https://cloud-native.slack.com there's a #buildpacks-mentoring channel. Someone there should be able to help you out.

[abhinandanudupa]

If you reach out on the project Slack -> https://cloud-native.slack.com there's a #buildpacks-mentoring channel. Someone there should be able to help you out.

@dmikusa unfortunately we cannot join the Slack community without a @rafay.co or @huawei.com email.

[dfreilich]

@abhinandanudupa That is probably another slack instance. You can get an invitation to the Cloud Native slack for any email here: https://communityinviter.com/apps/cloud-native/cncf

[abhinandanudupa]

@dfreilich Thank you!

[natalieparellano]

Here is a Miro board that we've used to illustrate how things work today and what a future workflow might be: https://miro.com/app/board/uXjVPrqGPB4=/

It might make sense to split this issue into two (one for buildpack packages, other for builders).

Buildpack packages

pack buildpack package

should look at the target data in buildpack.toml and create a different package for each os/arch combination that is supported by the buildpack. Optionally,

pack buildpack package --platform XXX

could create a package for the desired platform.

Builders

pack create builder --config builder.toml

Where builder.toml contains a reference to a build-time base image and runtime base image:

# This image is used at runtime
run-image = "cnbs/sample-stack-run:bionic"
# This image is used at build-time
build-image = "cnbs/sample-stack-build:bionic"

Note this schema will be modified as part of #1303.

If the referenced build-image and the referenced run-image are manifest lists, pack should create a different builder for each os/arch combination that is supported by all buildpacks. Optionally,

pack create builder --config builder.toml --platform XXX

should create a builder for the desired platform, failing if all buildpacks do not support it. Where builder.toml [[buildpacks]] references buildpack packages, those packages should be manifest lists (see #1460).

cc @AidanDelaney @jabrown85 @jjbustamante @joe-kimmel-vmw @jpena-r7 @ryanmoran

[jjbustamante]

Also, here is a recording of the meeting where we were talking using the mirror board mentioned by @natalieparellano

[AnirudhBot]

Hey @jjbustamante , I am really interested in this discussion and proposed solutions. I went through the buildpacks concepts and also the miro board link to get an understanding of pack build and the platform specification workflow. I would love to be a value contributor to this issue as a LFX mentee and eventually many more! Looking forward :)

[jjbustamante]

Now that the stack removal was shipped, this issue requires a sub-team RFC to clarify how we can implement the requirement now that targets are part of the buildpack.toml

[jjbustamante]

@dmikusa Do you have time to work on this? or do you want someone else to pick it up?

@hone I think this is related to what you mentioned in the last WG, how do we set the arch/os during creation of buildpackage

[dmikusa]

Happy to help in terms of discussion & testing, I don't have time at the moment to help with dev work.

[jjbustamante]

blocked on the following RFC

[edmorley]

I think the milestone for this issue is incorrect? It's currently in 0.35.0 but #2086 landed in 0.34.0.