Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for rebase omitting os/arch config properties #752

Merged

Conversation

micahyoung
Copy link
Contributor

@micahyoung micahyoung commented Aug 4, 2020

Fixes [#751] for missing os, architecture and os.version properties after mutate.Rebase

  • os, architecture properties are required by config spec [1]

  • os.version is unspec'd but required for Windows images to run on Windows containers [2]

This implementation takes these three properties from newBase, which most accurately defines the newly rebased image's os/architecture. If the properties were taken from original, it would prevent reuse of de-facto ABI-compatible top layers with differing old and new base images (i.e. data-only top layers rebased from amd64 to arm, Windows os.version upgrades).

This does not address additional, optional config properties that are currently being omitted during rebase such as author and created, which I feel could be addressed in a separate issue.

* Quick fix for missing `os` and `architecture` config properties, required by config spec [1]

* Also include unspec'd `os.version` property which is required for Windows images to run on Windows containers [2]

This implementation takes these three properties from newBase, which most accurately defines the newly rebased image's os/architecture. If the properties were taken from original, it would prevent reuse of de-facto ABI-compatible top layers with differing old and new base images (i.e. data-only top layers rebased from amd64 to arm, Windows os.version upgrades).

This does not address additional, optional config properties that are currently being omitted during rebase such as `author` and `created`.

Fixes [google#751]

[1]: https://github.com/opencontainers/image-spec/blob/master/config.md#properties
[2]: https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility#matching-container-host-version-with-container-image-versions

Signed-off-by: Micah Young <ymicah@vmware.com>
@micahyoung micahyoung force-pushed the fix-rebase-config-platform-properties branch from 0a59b66 to c8eb274 Compare August 4, 2020 17:47
@codecov-commenter
Copy link

codecov-commenter commented Aug 4, 2020

Codecov Report

Merging #752 into master will decrease coverage by 0.04%.
The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #752      +/-   ##
==========================================
- Coverage   79.06%   79.02%   -0.05%     
==========================================
  Files         102      102              
  Lines        4725     4734       +9     
==========================================
+ Hits         3736     3741       +5     
- Misses        548      550       +2     
- Partials      441      443       +2     
Impacted Files Coverage Δ
pkg/v1/mutate/rebase.go 50.74% <50.00%> (+0.74%) ⬆️
pkg/v1/remote/transport/error.go 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38ad4ec...c8eb274. Read the comment docs.

@micahyoung micahyoung marked this pull request as ready for review August 4, 2020 18:33
@jonjohnsonjr
Copy link
Collaborator

If the properties were taken from original, it would prevent reuse of de-facto ABI-compatible top layers with differing old and new base images (i.e. data-only top layers rebased from amd64 to arm, Windows os.version upgrades).

Does this describe your current use case or is it just hypothetical?

I see now that my original comments might not have made much sense. We want, largely, the config from the original image to remain intact, and this change takes the os/arch from the new base such that you can re-use top layers across different architectures.

I'm wondering if this isn't universally true? For example, what if I want to use gcr.io/distroless/static as a base image for multiple architectures of a go binary? There's nothing platform-specific about the gcr.io/distroless/static base layers, and the platform-specific bits would be in the top layers (just the binary). This seems like the opposite scenario from what you've described.

I think we're looking at two separate issues:

  1. Rebase drops all fields from ConfigFile other than Config, which is unexpected.
  2. How do we handle conflicts between the original image and the new base image? This ties into crane rebase : problems with environment variables #730 with environment variables, a bit.

Given that we currently don't set any platform info at all, I'm fine merging this and letting the new base image "win" for platform stuff, since you'd demonstrated a need for it, but we probably want some way to handle these conflicts e.g. to support the scenario I described.

@imjasonh does what I'm saying here make sense? Do you have any problem with just merging this? I don't think there's any backward-compat issues.

@micahyoung micahyoung changed the title Fix for rebase omitting os/arch config properties (#751) Fix for rebase omitting os/arch config properties Aug 5, 2020
@micahyoung
Copy link
Contributor Author

micahyoung commented Aug 5, 2020

Hey @jonjohnsonjr I'd be happy to use original for the source of these fields if it makes more sense to start, then later we can implement handling for resolving configfile property conflicts. It will let me remove the workaround in my own codebase and in its place, I can handle cherry-picking the conflicting values from new base after mutate.Rebase and letting everything else come from original.

I'm without power at home after a big storm in the eastern US so I wouldn't be able to adjust the PR until later today.

@jonjohnsonjr
Copy link
Collaborator

I'd be happy to use original for the source of these fields if it makes more sense to start, then later we can implement handling for resolving configfile property conflicts.

I thought about this a little bit more and I think the scenario I described is rather unlikely. Most base images contain platform-specific stuff, and distroless is a strange exception, so I'm content to merge this PR. I think it's a reasonable default, and we can provide other ways to achieve what I described.

It will let me remove the workaround in my own codebase and in its place, I can handle cherry-picking the conflicting values from new base after mutate.Rebase and letting everything else come from original.

Let's merge this to unblock you downstream, but some follow-up that might be worthwhile:

I'm without power at home after a big storm in the eastern US so I wouldn't be able to adjust the PR until later today.

Sorry to hear that :( hope you're doing alright.

Thanks for the PR!

@jonjohnsonjr jonjohnsonjr merged commit 6d4814a into google:master Aug 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants