Skip to content
This repository has been archived by the owner on Jan 22, 2024. It is now read-only.

CUDA / Docker & GPG error #658

Closed
pfriesch opened this issue Mar 9, 2018 · 12 comments
Closed

CUDA / Docker & GPG error #658

pfriesch opened this issue Mar 9, 2018 · 12 comments

Comments

@pfriesch
Copy link

pfriesch commented Mar 9, 2018

I have a similar problem to #619

FROM nvidia/cuda:9.0-cudnn7-runtime-ubuntu16.04
> apt-get update
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:3 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
Ign:5 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  InRelease
Get:6 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Release [564 B]
Get:7 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Release.gpg [801 B]
Get:8 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Packages [17.9 kB]
Err:8 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Packages
  Hash Sum mismatch
Fetched 18.2 kB in 0s (41.2 kB/s)
Reading package lists... Done
E: Failed to fetch http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/Packages.gz  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

> apt-key adv --verify /var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release.gpg /var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release
Executing: /tmp/tmp.X0hywaUFv0/gpg.1.sh --verify
/var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release.gpg
/var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release
gpg: Signature made Thu Feb 15 01:23:48 2018 UTC using RSA key ID 7FA2AF80
gpg: Good signature from "cudatools <cudatools@nvidia.com>"
gpg: WARNING: Using untrusted key!

I tried rm /etc/apt/sources.list.d/cuda.list -> same result
@Abai
Copy link

Abai commented Mar 9, 2018 •
edited
Loading

I think my problem is relevant to @Isn0gud

Started happening since this morning:

Step 7/8 : RUN apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y -q     libcudnn6=$CUDNN_VERSION-1+cuda8.0     libcudnn6-dev=$CUDNN_VERSION-1+cuda8.0  && rm -rf /var/lib/apt/lists/*
 ---> Running in 94b890e74e5a
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Ign:2 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64  InRelease
Get:3 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Ign:4 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  InRelease
Get:5 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64  Release [564 B]
Get:6 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Release [564 B]
Get:7 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64  Release.gpg [801 B]
Get:8 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [73.2 kB]
Get:9 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Release.gpg [801 B]
Get:10 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64  Packages [98.4 kB]
Get:11 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [589 kB]
Get:12 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Packages [17.9 kB]
Err:12 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64  Packages
  Hash Sum mismatch
Get:13 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]

Get:14 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]

Get:15 http://archive.ubuntu.com/ubuntu xenial/universe Sources [9,802 kB]
Get:16 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [12.7 kB]
Get:17 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [406 kB]
Get:18 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3,486 B]

Get:19 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,558 kB]
Get:20 http://archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [14.1 kB]

Get:21 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [9,827 kB]

Get:22 http://archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [176 kB]
Get:23 http://archive.ubuntu.com/ubuntu xenial-updates/universe Sources [242 kB]
Get:24 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [957 kB]
Get:25 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [13.1 kB]
Get:26 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [766 kB]
Get:27 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [18.5 kB]
Get:28 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [5,153 B]
Get:29 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [7,705 B]
Fetched 25.1 MB in 9s (2,576 kB/s)

Reading package lists...
E: Failed to fetch http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/Packages.gz  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

@flx42
Copy link
Member

flx42 commented Mar 9, 2018

Where you using the latest images? i.e. did you do a docker pull?
Are you also possibly missing a rm -rf /var/lib/apt/lists/* somewhere in your Dockerfiles?

@pfriesch
Copy link
Author

It was a fresh instance from today, so yeah they are the latest images. So it is basically a fresh install with only the latest docker in the ubuntu repo. I never called rm -rf /var/lib/apt/lists/*. Is that the expected way of using the docker images?

@flx42
Copy link
Member

flx42 commented Mar 10, 2018

No that's not the expected way, I was just trying to figure out what's wrong.
Given the log from @Abai above, I believe you are downloading a stale file from the repository, but I'm not sure why. I've circulated the information internally.

@pfriesch
Copy link
Author

The issue seems to be resolved on my end.

The GPG Key ist still untrusted though. But I do not have experience with packaging, so I don't know if that is even a problem.

> apt-key adv --verify /var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release.gpg /var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release
Executing: /tmp/tmp.NsvC45wcij/gpg.1.sh --verify
/var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release.gpg
/var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_Release
gpg: Signature made Thu Feb 15 01:23:48 2018 UTC using RSA key ID 7FA2AF80
gpg: Good signature from "cudatools <cudatools@nvidia.com>"
gpg: WARNING: Using untrusted key!

@Abai
Copy link

Abai commented Mar 10, 2018

@flx42 I was building a cuda runtime image that is almost identical to the official one at dockerhub. Only mine is based off another base image in the FROM directive.

@flx42
Copy link
Member

flx42 commented Mar 13, 2018

@Abai is it solved for you too?
I think it was probably some flakiness from our CDN, or something like that...

@Abai
Copy link

Abai commented Mar 13, 2018

@flx42

Yes! All looks good now. Thanks.

@selsius selsius mentioned this issue Mar 13, 2018
Closed
@np-csu
Copy link

np-csu commented May 30, 2018

you may delete/move two files:
RUN rm /etc/apt/sources.list.d/cuda.list && rm /etc/apt/sources.list.d/nvidia-ml.list
to solve the problem directly.

@flx42
Copy link
Member

flx42 commented May 30, 2018

@np-csu try using https in these files also

@davinwang
Copy link

@flx42 I solved the GPG issue by setting the /etc/hosts to assign developer.download.nvidia.com to 192.229.211.70, in order to override the default value 183.214.10.141/142 for me.

@liuchintao
Copy link

@flx42 I solved the GPG issue by setting the /etc/hosts to assign developer.download.nvidia.com to 192.229.211.70, in order to override the default value 183.214.10.141/142 for me.

@davinwang yes it worked, and it's the only successful method I tried :(

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@Abai @pfriesch @flx42 @davinwang @np-csu @liuchintao