Support for multi-platform artifacts when creating builders and buildpack package

[jjbustamante]

readable

Fixes pack #1459

[buildpack-bot]

Maintainers,

As you review this RFC please queue up issues to be created using the following commands:

/queue-issue <repo> "<title>" [labels]...
/unqueue-issue <uid>

Issues

(none)

[dmikusa]

Similar thoughts here. Could this be inferred based on the presence of multiple targets? I get that we don't want to just suddenly start making image indexes as that changes things. But couldn't we only do this if multiple targets with different arch's are specified? Or is there a case where you'd have multiple target/arch's and still not want that behavior?

[natalieparellano]

I am trying to think of who would be specifying multiple targets today and I'm guessing it would just be bash buildpacks. Maybe we could print a warning if multiple targets are defined but there is only one ./bin/detect or ./bin/build.

[jjbustamante]

The scenarios I was thinking were these:

1. I do not have targets on my buildpack.toml
   • In this case, the idea is to pack buildpack package do the same thing as it is doing today.
2. I do have targets on my buildpack.toml

• Daemon case: pack should build my buildpack package with the platform defined by the daemon (as it is doing today) without requiring any new flag.
• Registry or File case:
  • If more than 1 target is defined: pack will create the Image Index, If I just want to create 1 single package, I will need to use --target flag to defined which one.
  • If only 1 target is available, nothing changes

[dmikusa]

So this command could be used if one hasn't updated to use targets yet? i.e. still using stacks?

What about buildpacks that are migrating from stacks to no-stacks and have both present? This use case concerns me the most, cause I know we (Paketo) will be in this state for a while.

[dmikusa]

I'd be interested to hear more in general about this state of migrating from stacks to no-stacks. What commands would use the old stack metadata? what commands would use the new target info? I'm assuming we'd keep the two synchronized, but just curious more about how this would all work.

Or am I overthinking this, and the net result (i.e. the images generated) will be the same regardless of stacks or no stacks?

[jjbustamante]

Good point, regarding the stack removal plan we are planning to support both scenarios (stacks and run, build images), based on your feedback the idea will be to clarify how a world with:

• stacks
• targets
• multi-arch support
  will work

[dmikusa]

Would anything change for a composite/meta buildpack? I'd assume that you would need to have multi-arch images for every buildpack in the composite buildpack. Otherwise, would it work the same as a regular component buildpack?

[jjbustamante]

That's correct, I will add a new example with the composite builpack, but we are assuming you'd need to have multi-arch images for every buildpack in the composite buildpack

[natalieparellano]

I am sorry for opening this can of worms, but technically "target" when used in the context of say CNB_TARGET_OS or CNB_TARGET_ARCH refers exclusively to the run image. Right now, all target data for the build image and the run image must match - but perhaps that won't always be the case? What will we do then?

[hone]

@natalieparellano what do you think we should use here?

[natalieparellano]

I am pretty sure that the current architecture will accommodate this. What would probably end up happening is that buildpack.toml would have a list of [[targets]] and the buildpack os/arch/bin directories would be a subset of that list. Then at build time (somehow, but probably in run.toml) you'd have a list of run images to fulfill all the os/arch combinations in [[targets]] and by iterating through that list, you have a multi-arch application image. The key is that the mapping between buildpack.toml [[targets]] and the buildpack os/arch/bin directories would not necessarily be 1:1, whereas today we assume it is.

[joshwlewis]

Will these two copy steps outlined happen in different image layers? It seems like the idea here is to save disk space for files that are common across architectures. To achieve that, I think you'd need an image layer with the common files, that images for each architecture could include in their layer hierarchy. Maybe this was the idea and I missed it. If so, maybe worth mentioning somewhere.

Otherwise, if common files are in the same image layer as the architecture-specific files, clients would essentially be downloading and storing the common files once for each architecture.

[joshwlewis]

Ok, I did find the answer a bit further on: https://github.com/buildpacks/rfcs/pull/295/files#diff-cf9fc6f40b9a670966257d2475741958b49fb15684dd37cf8019514fe779885bR799. It looks like common and architecture specific files will be in one image layer.

[joshwlewis]

If the common files are duplicated across architecture images anyway, does the common file inheritance have much value? End users of buildpacks and builders won't really get much benefit. And for the CNBs I work on (Heroku's), they won't have many (maybe 0) files shared across architectures.

Just thinking common file inheritance is a heavy feature with some potential gotchas, and buildpack authors who want/need to dedupe files could probably get pretty far just by symlinking or hardlinking files into multiple {os}/{arch} folders.

[jjbustamante]

Hi @joshwlewis

Thank you very much for your feedback! I think that at least the buildpack.toml and the package.toml will be two common files shared across architectures, that's actually a win for a Buildpack Authors, right? because they do not need to keep separate files for each os/arch they want to support.

[natalieparellano]

common file inheritance is a heavy feature with some potential gotchas

I agree. Maybe it makes sense to take it out of this RFC and add it later if buildpack authors want it

[natalieparellano]

I think that at least the buildpack.toml and the package.toml will be two common files shared across architectures

I feel like buildpack.toml could be the one and only special case. IDK if package.toml is something that we even expect to be inside a buildpack root directory at build time. It's just easier to comprehend the packaging step if we don't move too much stuff around inside the directory tree. Perhaps we could spec out that we'd resolve symlinks when creating the layer if folks want to take advantage of that.

[jjbustamante]

Well, if I take a look at some buildpacks for Paketo for example, they have an structure like this one in the root folder:

├── LICENSE
├── NOTICE
├── README.md
├── buildpack.toml
├── package.toml
├── resources
│ ├── config.properties

In these cases, I think they want to shared those files on each platform images right?

[joshwlewis]

In these cases, I think they want to shared those files on each platform images right?

They could copy, symlink, or hard link files into each platform folder. I don't think that's a big challenge for buildpack authors (they probably script this in some way anyway). Honestly, I think it might be a bit more straightforward for them to understand, too. Copy, symlink, and hardlink are established concepts; they won't need to read this RFC or the buildpack spec to know how to do that.

I still think there's convenience in this shared file logic. I'm mostly just thinking you could simplify the initial implementation, long term maintenance, and probably this RFC without it.

[natalieparellano]

I like to think of --target as more of a helpful way to narrow down the os/arch when buildpack.toml has multiple definitions, or to avoid creating a manifest list. Is that correct?

[jjbustamante]

Yes, exactly when you actually want to avoid creating multiple images, then target will help you

[natalieparellano]

This is how it works today, but this feels like a bug. The OCI image spec says this field is required.

[jjbustamante]

Maybe it is a bug, but apparently nobody is complaining about it! I didn't want to start filling this attribute and break the current behavior for end-users.

[natalieparellano]

Actually this did come up recently (here and here) I think we said this RFC would fix those cases but maybe we assumed first the buildpack author would upgrade to use targets. In any case, it probably doesn't hurt... I'm fine to leave it out of this RFC though

[jjbustamante]

Yeah, I think is what @jericop is trying to fix also here

[natalieparellano]

This assumes that the build image is a manifest list, no? Because if the build image is linux/arm64 (only) then we create an arm64 image, no?

[jjbustamante]

Yeah, the pack builder create command assumes the build, run and buildpack images are multi-arch, and when building for a particular platform, pack should validate it

[natalieparellano]

Awesome work @jjbustamante. I would really prefer if we kept "shared" files out of this RFC, it would make it a lot simpler to understand and implement. But otherwise this looks great.

[jjbustamante]

This is actually the current messaged showed in pack

[edmorley]

@jjbustamante I believe this should be [[targets.distros]] plus versions should be version (singular) and a string not an array?

(Plus the same change made to the 3-4 similar references in this file)

[schneems]

I'm wondering what automated checks around non-code PRs such as a buildpack.toml linter (for example) might look like.

[jjbustamante]

@jjbustamante I believe this should be [[targets.distros]] plus versions should be version (singular) and a string not an array?

(Plus the same change made to the 3-4 similar references in this file)

Yes!! You are right @edmorley! I will ask how could I do the fix now that this was merged

[jjbustamante]

I just saw you opened a PR to fix a previous RFC, is it possible for you to open one for this?

[edmorley]

Sure - I've opened #312.

[jjbustamante]

Thank you!! @edmorley