Multi-stage COPY --from should preserve ownership/permissions

[thaJeztah]

This relates to:

• Support "--from" for ADD in multi stage build #36759 "Support "--from" for ADD in multi stage build"
• COPY --from=foo bar.tgz ... #37112 COPY --from=foo bar.tgz ...

Problem statement

The COPY (and ADD) Dockerfile instructions by default reset the ownership of files added to 0:0.

While this makes sense when copying files from a build-context (users/groups on the host in most situations won't match user/group in the container), in multi-stage builds this situation may be different.

In a multi-stage build, intermediate stages are meant to prepare content/artifacts for the final stage(s) they are copied to. This preparation can include: setting the correct ownership (and permissions) of files.

Because of the current behavior of COPY, those permissions are reset, and workarounds, such as tar-ing the files before COPY-ing, then extracting the tar in the final stage (which preserves permissions and ownership as set on the files before tar-ing) are not ideal.

Proposal

I propose to preserve permissions and ownership of files/directories when COPY-ing between stages in a multi-stage build

Example

Building this Dockerfile on a current version of Docker:

FROM busybox AS one
RUN mkdir -p /foo/1-subdir \
 && touch \
  /foo/4-five-six \
  /foo/7-eight-nine \
 && chown -R 123:123 /foo/1-subdir \
 && chown 456:456 /foo/4-five-six \
 && chown 789:789 /foo/7-eight-nine \
 && chmod -R 0600 /foo/1-subdir \
 && chmod 0060 /foo/4-five-six \
 && chmod 0006 /foo/7-eight-nine
RUN echo "In stage one" \
 && ls -l /foo/


FROM busybox AS final
COPY --from=one /foo /bar
RUN echo "In final stage" \
 && ls -l /bar/

Produces:

In stage one
total 4
drw-------    2 123      123           4096 May 22 12:24 1-subdir
----rw----    1 456      456              0 May 22 12:24 4-five-six
-------rw-    1 789      789              0 May 22 12:24 7-eight-nine

In final stage
total 4
drw-------    2 root     root          4096 May 22 12:24 1-subdir
----rw----    1 root     root             0 May 22 12:24 4-five-six
-------rw-    1 root     root             0 May 22 12:24 7-eight-nine

With the proposed changes, the final stage would look like:

In final stage
total 4
drw-------    2 123      123           4096 May 22 12:24 1-subdir
----rw----    1 456      456              0 May 22 12:24 4-five-six
-------rw-    1 789      789              0 May 22 12:24 7-eight-nine

Question / to be discussed

COPY --from accepts both the name/number of a build-stage, as well as an image-reference:

• Should we preserve ownership/permissions when copying from an image as well? (COPY --from myimage:latest)
• Should we add new options to make the --from less ambiguous, and only preserve ownership/permissions when copying from other stages (i.e., add --from-stage and --from-image options)?

[tonistiigi]

Should we preserve ownership/permissions when copying from an image as well? (COPY --from myimage:latest)

Yes. I don't see any difference and user can always override.

[elibarzilay]

Note that there are other things that tar is helping with, like preserving set[ug]id, time stamps, hard links, etc.

BTW, at some point in the distant past it was common to tar cf foo | (cd somewhere; tar xf) as a way of doing what is now cp -a. From a definition POV of what is preserved, throwing back to "whatever tar does" might be a little more convenient than keeping your own list.

[elygre]

Yes to this change! I'm looking to reduce the size of the Oracle database docker images, and using a multi stage builds saves around 9 GB (!). However, it cannot reliably work: File ownerships in the installation stage are set by an installation program, and the Dockerfile cannot know what they should be. Having "COPY --from" maintain current file attributes would sove this; adding a "--same-owner" or similar will also be a fine solution.

[thaJeztah]

Related; #37830

[nedix]

One terrible implication is that users lose access to their home directory. Proposal seems good. Related: #32816

[tonistiigi]

This has been fixed in buildkit.

[thaJeztah]

@tonistiigi should we make the same change for non-buildkit? (not sure how easy it is to implement in the classic builder?)

[tonistiigi]

@thaJeztah I'm ok with that.

[huggla]

This bug concerned both ownership and permissions while #38599 only mentions ownership. Are both ownership and permissions preserved now, or only ownership?