-
Notifications
You must be signed in to change notification settings - Fork 18.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Digest is lost after loading a saved image #22011
Comments
ping @aaronlehmann ptal |
The digest field is only populated for images that are pulled by digest. So loading an image from a tar won't fill it. I'm not sure the field could be carried through on save/load because of the security implications. A saved image could lie about the manifest's digest, since the manifest is not part of the saved image. Generally, it's better to use use the image ID to check image provenance. The image ID is a secure hash over the image itself, whereas the |
Thanks for the clarification @aaronlehmann, few follow ups:
|
We probably should, but IIRC there are some user interface issues around these digests that should be cleared up first.
If you save and load an image, the ID shouldn't change, so it should be possible to use it in this situation.
The manifest references compressed layers. The layers don't get compressed until We've explored compressing layers when they are first created to make it possible for manifests to be persistent like you suggest, but there are two issues with doing this:
Frankly, it's a tradeoff that we've always struggled with. What you're suggesting would definitely have advantages for extending the content trust chain to subsequent pushes and saves. But the disadvantages I mentioned would probably cause some backlash. |
Thanks for the clarifications @aaronlehmann.
|
That's strange. @tonistiigi do you know why this would happen?
This is essentially how
Unfortunately there isn't, because the registry operates in terms of compressed layers. |
Tested saving alpine latest on v1.10 and loading to empty v1.11. ID was the same. |
I'm sorry @tonistiigi, my machines were running Docker v1.9.1 and Docker v1.11.1 (also reproduced the issue on clean machines). |
@liron-l This is expected. |
This also surprised me at first -- I've started looking into the content adressability changes form 1.10 wanted to check all the IDs myself locally. A bit of background that might be useful if you don't work closely with the docker codebase:
The kicker is that as other people have mentioned is the the distribution digest sums over compressed tar files which the docker engine doesn't store, and the exact size of it depends on the compression settings used. (And possibly other things like what version of libz the client has?) It's not as simple as including the compression settings use to generate the compressed tar so that a client could reconstruct things is it? |
Just stumbled across the same problem. We have to use docker save/load to distribute Docker images as we don't have access to the internet on the destination hosts. We also have to make sure that the images on the destination hosts end up being exactly the same that were used while bundling the application. Content addressable images sounded perfect for this, but due to this issue this does not work as we hoped. |
Just a simple workaround.[Only work when the computer can access repo] write a simple Dockerfile as
and run Then, the digest will be generated. @thaJeztah Along with docker 17.03.0-ce, service create logic uses the image digest and this issue would be quite important for those who are using save and load command. |
ping @nishanttotla @aaronlehmann (this came up in a discussion recently) |
Using `:latest` tag is typically a cause of pain, as underlying image changes behavior. Here, I'm switching to using a point release, and re-updating the stored tarballs with it. Sadly, when saving/loading images, the repo digeset is not supported: moby/moby#22011 ; but using point releases should mitigate the problem. The motivation here is that docker tests have some flakiness due to accidental importing of `busybox:latest` which has `/bin/nc` that no longer supports `-p 0`: ``` $ docker run -it --rm busybox /bin/nc -l 127.0.0.1 -p 0 Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox Digest: sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812 Status: Downloaded newer image for busybox:latest nc: bad local port '0' ``` Looks like older busybox versions (e.g. `busybox:1.24` do honor `-p 0` as the test expect, but I would rather update busybox to fix.
Using `:latest` tag is typically a cause of pain, as underlying image changes behavior. Here, I'm switching to using a point release, and re-updating the stored tarballs with it. Sadly, when saving/loading images, the repo digeset is not supported: moby/moby#22011 ; but using point releases should mitigate the problem. The motivation here is that docker tests have some flakiness due to accidental importing of `busybox:latest` which has `/bin/nc` that no longer supports `-p 0`: ``` $ docker run -it --rm busybox /bin/nc -l 127.0.0.1 -p 0 Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox Digest: sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812 Status: Downloaded newer image for busybox:latest nc: bad local port '0' ``` Looks like older busybox versions (e.g. `busybox:1.24` do honor `-p 0` as the test expect, but I would rather update busybox to fix.
Using `:latest` tag is typically a cause of pain, as underlying image changes behavior. Here, I'm switching to using a point release, and re-updating the stored tarballs with it. Sadly, when saving/loading images, the repo digeset is not supported: moby/moby#22011 ; but using point releases should mitigate the problem. The motivation here is that docker tests have some flakiness due to accidental importing of `busybox:latest` which has `/bin/nc` that no longer supports `-p 0`: ``` $ docker run -it --rm busybox /bin/nc -l 127.0.0.1 -p 0 Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox Digest: sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812 Status: Downloaded newer image for busybox:latest nc: bad local port '0' ``` Looks like older busybox versions (e.g. `busybox:1.24` do honor `-p 0` as the test expect, but I would rather update busybox to fix.
Using `:latest` tag is typically a cause of pain, as underlying image changes behavior. Here, I'm switching to using a point release, and re-updating the stored tarballs with it. Sadly, when saving/loading images, the repo digeset is not supported: moby/moby#22011 ; but using point releases should mitigate the problem. The motivation here is that docker tests have some flakiness due to accidental importing of `busybox:latest` which has `/bin/nc` that no longer supports `-p 0`: ``` $ docker run -it --rm busybox /bin/nc -l 127.0.0.1 -p 0 Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox Digest: sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812 Status: Downloaded newer image for busybox:latest nc: bad local port '0' ``` Looks like older busybox versions (e.g. `busybox:1.24` do honor `-p 0` as the test expect, but I would rather update busybox to fix.
Using `:latest` tag is typically a cause of pain, as underlying image changes behavior. Here, I'm switching to using a point release, and re-updating the stored tarballs with it. Sadly, when saving/loading images, the repo digeset is not supported: moby/moby#22011 ; but using point releases should mitigate the problem. The motivation here is that docker tests have some flakiness due to accidental importing of `busybox:latest` which has `/bin/nc` that no longer supports `-p 0`: ``` $ docker run -it --rm busybox /bin/nc -l 127.0.0.1 -p 0 Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox Digest: sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812 Status: Downloaded newer image for busybox:latest nc: bad local port '0' ``` Looks like older busybox versions (e.g. `busybox:1.24` do honor `-p 0` as the test expect, but I would rather update busybox to fix.
Using `:latest` tag is typically a cause of pain, as underlying image changes behavior. Here, I'm switching to using a point release, and re-updating the stored tarballs with it. Sadly, when saving/loading images, the repo digeset is not supported: moby/moby#22011 ; but using point releases should mitigate the problem. The motivation here is that docker tests have some flakiness due to accidental importing of `busybox:latest` which has `/bin/nc` that no longer supports `-p 0`: ``` $ docker run -it --rm busybox /bin/nc -l 127.0.0.1 -p 0 Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox Digest: sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812 Status: Downloaded newer image for busybox:latest nc: bad local port '0' ``` Looks like older busybox versions (e.g. `busybox:1.24` do honor `-p 0` as the test expect, but I would rather update busybox to fix.
Is there any progress on this? It's blocking our workflow. We're trying to develop an offline installer for our app, but some of the images we use are specified by digest. |
This is no longer blocking us, because we've found a workaround, but it's a bit of a hack:
|
I want to pull When I run So if it is possible when I docker pull |
Since image ids are now stable would it be possible to allow us to do @ in docker compose so that we wouldn't need to pull the digests through? |
Output of
docker version
:Output of
docker info
:Additional environment details (AWS, VirtualBox, physical, etc.):
physical
Steps to reproduce the issue:
docker pull "alpine@sha256:4f2d8bbad359e3e6f23c0498e009aaa3e2f31996cbea7269b78f92ee43647811"
docker images --digests
docker save alpine@sha256:4f2d8bbad359e3e6f23c0498e009aaa3e2f31996cbea7269b78f92ee43647811 > test.tar
docker rmi alpine@sha256:4f2d8bbad359e3e6f23c0498e009aaa3e2f31996cbea7269b78f92ee43647811
docker load < test.tar
docker images --digests
also indocker inspect
.Describe the results you received:
Digest disappear (and
RepoTags
andRepoDigests
are empty).Describe the results you expected:
Repo tags and digests remain the same
Additional information you deem important (e.g. issue happens only occasionally):
The text was updated successfully, but these errors were encountered: