Update Distribution Spec for Windows vs Linux Buildpackages

[ameyer-pivotal]

Updates layer expectations for Windows buildpackages

Signed-off-by: Andrew Meyer meyeran@vmware.com

---

This change accommodates Windows buildpackages, which currently require a base image in order to be pulled into a local docker daemon. This may change in the future, and the clarification to the spec will allow for this flexibility, since it will no longer include any potential base layers in the contract.

Also, regarding the Windows specific blob format (where tar paths must be prefixed with Files/), see the last paragraph of this spec page, as it is a blanket statement that covers all of the spec. Therefore we did not add the prefix to this dist spec specifically.

[ekcasey]

I don't understand this change. Don't we dictate that every layer in a buildpackage MUST be a buildpack. When is there a layer in a buildpackage that is not intended to be a buildpack?

[ameyer-pivotal]

@ekcasey This is described in more detail in the PR description, but the short version is: Windows buildpackages have base/OS layers that are not buildpacks. Even the smallest viable base, though small, is not exactly equivalent to scratch on linux.

[ekcasey]

@ameyer-pivotal thanks for the clarification, I knew runnable windows images needed some sort of base layer, but I was unaware that even docker pull required one!

[ekcasey]

I worry that the wording may be confusing to readers without the explicit windows context. Do we want to say something like "Each FS layer blob must contain a single buildpack with the following exception.. . Each buildpack layer MUST...".

[ameyer-pivotal]

@ekcasey What exception would be specified? OS layers? The approach I was taking is to have the spec not specifically mention any layers that aren't in the contract (i.e. not mention non-buildpack layers). Since platforms consuming buildpackages only look at buildpack layers, the author of a buildpackage should technically be able to put whatever they want in any other layers, as they would not be "under contract/spec."

That said, I'm willing to budge if it helps. Perhaps a simple change to use non-OS layer blob instead of FS layer blob in the original wording (though that might bring up ambiguity about what an OS layer is/isn't). This would give Each non-OS layer blob in the buildpackage MUST... Thoughts? The more I re-read that, the more I feel we'd go down a rabbit hole if we start mentioning other layer types. My stance is that the dist spec should ONLY comment on buildpack layers because that's all it cares about.

[hone]

@ameyer-pivotal can we put this context in your git commit as well? This clarification has been super insightful.

[sclevine]

After chatting with @ekcasey @nebhale and @hone, we'd suggest adding an exception for Windows, as opposed to opening up the buildpackage format to arbitrary layers.

[ameyer-pivotal]

@sclevine Actually, the reason for the original approach is that Windows buildpackages may not always look this way. We didn't want to cause a change to the spec if we're ever able to have a scratch equivalent for Windows images in the future (currently, scratch-based images work when packages are remote-only -- it's just pulling with docker that causes the issue). I still think that the spec should comment only on the parts it cares about. How would additional layers impact the platform's ability to look for the layers it's looking for? I.e. what do we gain by being restrictive?

Perhaps I should have gone with an RFC after all, as it seems like there is still some conversation to be had. I had understood the original suggested change to be a simple clarification of the original intention of the spec (all about buildpack layers), but adding exceptions for specific OSes feels like a more substantive change to the spec, rather than a clarification of the original spec.

(cc: @ekcasey @nebhale @hone)

[ameyer-pivotal]

@ekcasey @sclevine @jkutner @hone @nebhale Made an update to the wording to make specific callouts for Linux vs Windows packages. After giving it some thought, I'm on board with avoiding opening up buildpackages to arbitrary layers. If Windows buildpackages no longer need OS layers in the future, we can address it then with an RFC if desired.