Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix multistage copy use cached layer, when it should not #853

Closed
wants to merge 1 commit into from

Conversation

duythinht
Copy link

Fixes #589, #845 and related multistage dockerfile copy command.

Due to multistage dockerfile bug, copy command should not use cached layer when copy a glob, which is result of previous stage.
My workaround is detected CopyCommand.FilesUsedFromContext does not resolve the context from previous stage. So the the command only caching by hash of cmd.String(), which constant itself. After all I make decision to cache only the copy command, which use the file context instead of every copy command from other stage.

Why is this the best approach?

Most of copy --from, is using layer from previous stage, those contains glob of compiled, compressed, and may be retrieved from cached layer before, so we just need to copy instead of trying fetch cache by itself.

  • Includes [unit tests]

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@duythinht
Copy link
Author

@googlebot I signed it!

@googlebot
Copy link

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@duythinht duythinht changed the title Fixed multistage copy use cached layer, when it shouldn't not Fix multistage copy use cached layer, when it shouldn't not Nov 10, 2019
@duythinht duythinht changed the title Fix multistage copy use cached layer, when it shouldn't not Fix multistage copy use cached layer, when it should not Nov 13, 2019
@abergmeier
Copy link
Contributor

Can we get this reviewed please?

@rearden-steel
Copy link

Please review it, this is a real problem for us.

@mcfedr mcfedr mentioned this pull request Nov 25, 2019
4 tasks
@cvgw
Copy link
Contributor

cvgw commented Nov 25, 2019

@duythinht thanks for the PR! However I think this change would prevent COPY --from commands from ever being cached. I think it would preferable to implement a fix that allows those commands to be correctly cached rather than avoiding caching all together. Apologies if I have misunderstood anything.

As a side note, I have also been exploring a fix for these issues. One possible solution is in #880

@rearden-steel
Copy link

rearden-steel commented Nov 25, 2019

May be a good solution will be to check if the layer which the file is copied from was cached or not and use or do not use cache for current layer?

@cvgw
Copy link
Contributor

cvgw commented Dec 17, 2019

I believe this issue is fixed and this PR can be closed.

@duythinht duythinht closed this Dec 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Multistage Dockerfile with copy and glob uses cached layer when it shouldn't
5 participants