-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for multi-platform artifacts when creating builders and buildpack package #295
Support for multi-platform artifacts when creating builders and buildpack package #295
Conversation
Maintainers, As you review this RFC please queue up issues to be created using the following commands:
Issues(none) |
3da0d9b
to
06118b6
Compare
- Add a new boolean `--multi-arch` flag to indicate pack it must create multiples OCI artifacts and combine them with an [image index](https://github.com/opencontainers/image-spec/blob/master/image-index.md), | ||
this flag must be used in conjunction with `--publish` or `--format file` flags and error out when `daemon` is selected. | ||
> **Note** | ||
> I got feedback to avoid creating an Image Index if users are not expecting that, the flag is an acknowledgment for creating the index |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar thoughts here. Could this be inferred based on the presence of multiple targets? I get that we don't want to just suddenly start making image indexes as that changes things. But couldn't we only do this if multiple targets with different arch's are specified? Or is there a case where you'd have multiple target/arch's and still not want that behavior?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am trying to think of who would be specifying multiple targets today and I'm guessing it would just be bash buildpacks. Maybe we could print a warning if multiple targets are defined but there is only one ./bin/detect or ./bin/build.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The scenarios I was thinking were these:
- I do not have
targets
on mybuildpack.toml
- In this case, the idea is to
pack buildpack package
do the same thing as it is doing today.
- In this case, the idea is to
- I do have
targets
on mybuildpack.toml
- Daemon case:
pack
should build my buildpack package with the platform defined by the daemon (as it is doing today) without requiring any new flag. - Registry or File case:
- If more than 1 target is defined:
pack
will create theImage Index
, If I just want to create 1 single package, I will need to use--target
flag to defined which one. - If only 1 target is available, nothing changes
- If more than 1 target is defined:
```bash | ||
pack buildpack package <buildpack> --config ./package.toml --publish --platform linux/arm64 --platform linux/amd64 --multi-arch | ||
A multi-arch buildpack package will be created for platforms: 'linux/amd64', 'linux/arm64' | ||
Successfully published package <buildpack> and saved to registry | ||
|
||
# Or | ||
pack buildpack package <buildpack> --config ./package.toml --format file --platform linux/arm64 --platform linux/amd64 --multi-arch | ||
A multi-arch buildpack package will be created for platforms: 'linux/amd64', 'linux/arm64' | ||
Successfully created package <buildpack> and saved to file | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this command could be used if one hasn't updated to use targets yet? i.e. still using stacks?
What about buildpacks that are migrating from stacks to no-stacks and have both present? This use case concerns me the most, cause I know we (Paketo) will be in this state for a while.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd be interested to hear more in general about this state of migrating from stacks to no-stacks. What commands would use the old stack metadata? what commands would use the new target info? I'm assuming we'd keep the two synchronized, but just curious more about how this would all work.
Or am I overthinking this, and the net result (i.e. the images generated) will be the same regardless of stacks or no stacks?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, regarding the stack removal plan we are planning to support both scenarios (stacks and run, build images), based on your feedback the idea will be to clarify how a world with:
- stacks
- targets
- multi-arch support
will work
|
||
## Buildpack Package | ||
|
||
As a quick summary, our current process to create a buildpack package involves: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would anything change for a composite/meta buildpack? I'd assume that you would need to have multi-arch images for every buildpack in the composite buildpack. Otherwise, would it work the same as a regular component buildpack?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's correct, I will add a new example with the composite builpack, but we are assuming you'd need to have multi-arch images for every buildpack in the composite buildpack
|
||
# List of targets operating systems, architectures and versions | ||
|
||
[[targets]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am sorry for opening this can of worms, but technically "target" when used in the context of say CNB_TARGET_OS
or CNB_TARGET_ARCH
refers exclusively to the run image. Right now, all target data for the build image and the run image must match - but perhaps that won't always be the case? What will we do then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@natalieparellano what do you think we should use here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am pretty sure that the current architecture will accommodate this. What would probably end up happening is that buildpack.toml would have a list of [[targets]]
and the buildpack os/arch/bin directories would be a subset of that list. Then at build time (somehow, but probably in run.toml) you'd have a list of run images to fulfill all the os/arch combinations in [[targets]]
and by iterating through that list, you have a multi-arch application image. The key is that the mapping between buildpack.toml [[targets]]
and the buildpack os/arch/bin directories would not necessarily be 1:1, whereas today we assume it is.
a650045
to
f2034f6
Compare
40e16aa
to
783522e
Compare
- For each `target` an OCI image will be created, following these rules | ||
- `pack` will copy everything in the root folder excluding all `{os}/**` folder | ||
- `pack` will determine the **target root binary folder**, this is the specific root folder for a given `target` (based on the `targets.path` in the buildpack.toml or inferring it from a folder structure similar to the one show above) | ||
- `pack` will copy everything from the **target root binary folder** into the base buildpack package folder, in case a conflict file name is found, `pack` will **override** the existing file with the latest one |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will these two copy steps outlined happen in different image layers? It seems like the idea here is to save disk space for files that are common across architectures. To achieve that, I think you'd need an image layer with the common files, that images for each architecture could include in their layer hierarchy. Maybe this was the idea and I missed it. If so, maybe worth mentioning somewhere.
Otherwise, if common files are in the same image layer as the architecture-specific files, clients would essentially be downloading and storing the common files once for each architecture.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I did find the answer a bit further on: https://github.com/buildpacks/rfcs/pull/295/files#diff-cf9fc6f40b9a670966257d2475741958b49fb15684dd37cf8019514fe779885bR799. It looks like common and architecture specific files will be in one image layer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the common files are duplicated across architecture images anyway, does the common file inheritance have much value? End users of buildpacks and builders won't really get much benefit. And for the CNBs I work on (Heroku's), they won't have many (maybe 0) files shared across architectures.
Just thinking common file inheritance is a heavy feature with some potential gotchas, and buildpack authors who want/need to dedupe files could probably get pretty far just by symlinking or hardlinking files into multiple {os}/{arch}
folders.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @joshwlewis
Thank you very much for your feedback! I think that at least the buildpack.toml
and the package.toml
will be two common files shared across architectures, that's actually a win for a Buildpack Authors, right? because they do not need to keep separate files for each os/arch they want to support.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
common file inheritance is a heavy feature with some potential gotchas
I agree. Maybe it makes sense to take it out of this RFC and add it later if buildpack authors want it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that at least the buildpack.toml and the package.toml will be two common files shared across architectures
I feel like buildpack.toml
could be the one and only special case. IDK if package.toml
is something that we even expect to be inside a buildpack root directory at build time. It's just easier to comprehend the packaging step if we don't move too much stuff around inside the directory tree. Perhaps we could spec out that we'd resolve symlinks when creating the layer if folks want to take advantage of that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, if I take a look at some buildpacks for Paketo for example, they have an structure like this one in the root folder:
├── LICENSE
├── NOTICE
├── README.md
├── buildpack.toml
├── package.toml
├── resources
│ ├── config.properties
In these cases, I think they want to shared those files on each platform images right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In these cases, I think they want to shared those files on each platform images right?
They could copy, symlink, or hard link files into each platform folder. I don't think that's a big challenge for buildpack authors (they probably script this in some way anyway). Honestly, I think it might be a bit more straightforward for them to understand, too. Copy, symlink, and hardlink are established concepts; they won't need to read this RFC or the buildpack spec to know how to do that.
I still think there's convenience in this shared file logic. I'm mostly just thinking you could simplify the initial implementation, long term maintenance, and probably this RFC without it.
|
||
We propose: | ||
- Add a new `--target` flag with the follwing format `[os][/arch][/variant]:[name@version]` to build for a particular target, once the `platform.os` field is removed, | ||
this will be the way for end-users to specify the platform for which they want to create single OCI artifact. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like to think of --target
as more of a helpful way to narrow down the os/arch when buildpack.toml has multiple definitions, or to avoid creating a manifest list. Is that correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, exactly when you actually want to avoid creating multiple images, then target
will help you
|
||
```json | ||
{ | ||
"architecture": "", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is how it works today, but this feels like a bug. The OCI image spec says this field is required.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it is a bug, but apparently nobody is complaining about it! I didn't want to start filling this attribute and break the current behavior for end-users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
```bash | ||
pack builder create <builder> --config ./builder.toml | ||
Warning: "stack" has been deprecated, prefer "targets" instead: https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md | ||
Warning: A new '--target' flag is available to set the target platform for the builder, using 'linux/amd64' as default |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This assumes that the build image is a manifest list, no? Because if the build image is linux/arm64 (only) then we create an arm64 image, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, the pack builder create
command assumes the build, run and buildpack images are multi-arch, and when building for a particular platform, pack
should validate it
Awesome work @jjbustamante. I would really prefer if we kept "shared" files out of this RFC, it would make it a lot simpler to understand and implement. But otherwise this looks great. |
```bash | ||
pack buildpack package <buildpack> --config ./package.toml --publish --target linux/arm64 --target linux/amd64 | ||
A multi-arch buildpack package will be created for target platforms: 'linux/amd64', 'linux/arm64' | ||
Successfully published package <buildpack> and saved to registry |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is actually the current messaged showed in pack
2a5741f
to
44b9cca
Compare
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
…e the path for binaries in buildpack.toml Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
… distribution spec Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Co-authored-by: Natalie Arellano <narellano@vmware.com> Signed-off-by: Juan Bustamante <bustamantejj@gmail.com> Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Co-authored-by: Natalie Arellano <narellano@vmware.com> Signed-off-by: Juan Bustamante <bustamantejj@gmail.com> Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
…ic when exporting multi-arch buildpack packages as files Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Co-authored-by: Rune Soerensen <runesoerensen@gmail.com> Signed-off-by: Juan Bustamante <bustamantejj@gmail.com> Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Co-authored-by: Rune Soerensen <runesoerensen@gmail.com> Signed-off-by: Juan Bustamante <bustamantejj@gmail.com> Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
44b9cca
to
abd3ba2
Compare
[[targets.distributions]] | ||
name = "windows" | ||
versions = ["10.0.20348.1970"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jjbustamante I believe this should be [[targets.distros]]
plus versions
should be version
(singular) and a string not an array?
(Plus the same change made to the 3-4 similar references in this file)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering what automated checks around non-code PRs such as a buildpack.toml
linter (for example) might look like.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jjbustamante I believe this should be
[[targets.distros]]
plusversions
should beversion
(singular) and a string not an array?(Plus the same change made to the 3-4 similar references in this file)
Yes!! You are right @edmorley! I will ask how could I do the fix now that this was merged
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just saw you opened a PR to fix a previous RFC, is it possible for you to open one for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure - I've opened #312.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!! @edmorley
Updates RFC 0128 to use the correct `[[targets.distros]]` schema as defined already in the spec: https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpacktoml-toml I've not added an `## Amended` section, since: 1. The RFC has only just merged (this is effectively a review comment that missed the merge by a day) 2. The parts of the RFC being fixed are existing concepts already in the spec, rather than design decisions relating to the purpose of the RFC itself. Fixes: buildpacks#295 (comment)
Updates RFC 0128 to use the correct `[[targets.distros]]` schema as defined already in the spec: https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpacktoml-toml I've not added an `## Amended` section, since: 1. The RFC has only just merged (this is effectively a review comment that missed the merge by a day) 2. The parts of the RFC being fixed are existing concepts already in the spec, rather than design decisions relating to the purpose of the RFC itself. Fixes: buildpacks#295 (comment) Signed-off-by: Ed Morley <501702+edmorley@users.noreply.github.com>
[buildpacks#295] Signed-off-by: Natalie Arellano <narellano@vmware.com>
Updates RFC 0128 to use the correct `[[targets.distros]]` schema as defined already in the spec: https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpacktoml-toml I've not added an `## Amended` section, since: 1. The RFC has only just merged (this is effectively a review comment that missed the merge by a day) 2. The parts of the RFC being fixed are existing concepts already in the spec, rather than design decisions relating to the purpose of the RFC itself. Fixes: buildpacks#295 (comment) Signed-off-by: Ed Morley <501702+edmorley@users.noreply.github.com>
readable
Fixes pack #1459