Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Disable pip cache when creating Docker images #8575

Merged
merged 1 commit into from
Jul 29, 2021
Merged

[CI] Disable pip cache when creating Docker images #8575

merged 1 commit into from
Jul 29, 2021

Conversation

leandron
Copy link
Contributor

@leandron leandron commented Jul 28, 2021

Disable pip cache when creating Docker images:

  • For each Python package we install in any docker/install/*.sh script, we re keeping a copy of it in /root/.cache/pip/*
  • Disabling the pip cache is a good practice to save storage space in the Docker images being created
  • Also sort pip package lists alphabetically

In summary, this saves around 700Mb on the ci_cpu image, and 1.6Gb on the ci_gpu image.

cc @tqchen @areusch @mbrookhart @Mousius for reviews

* This is a good practice to save storage space in
  the Docker images being created
* Also sort pip package lists alphabetically
@leandron leandron requested review from areusch and jroesch as code owners July 28, 2021 16:44
Copy link
Contributor

@areusch areusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @leandron ! just one question for you

@@ -23,6 +23,9 @@ FROM ubuntu:18.04
RUN apt-get update --fix-missing
RUN apt-get install -y ca-certificates gnupg2

# Globally disable pip cache
RUN pip config set global.cache-dir false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it global.no-cache-dir or global.cache-dir?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it locally and it seems to work to set the global config and preventing the cache to be created.

When using it in an individual pip install —no-cache-dir foo it seems to be the way to disable in one instance, but I decided not going this way, because there is always a new pip install in those scripts.

Copy link
Contributor

@areusch areusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @leandron!

@areusch areusch merged commit cb395ff into apache:main Jul 29, 2021
mbaret pushed a commit that referenced this pull request Aug 2, 2021
* The fix to disable cache needs to run after pip is installed
* This is quick follow up fix after #8575
@leandron leandron deleted the docker_image_size branch September 8, 2021 11:57
ylc pushed a commit to ylc/tvm that referenced this pull request Sep 29, 2021
* This is a good practice to save storage space in
  the Docker images being created
* Also sort pip package lists alphabetically
ylc pushed a commit to ylc/tvm that referenced this pull request Sep 29, 2021
* The fix to disable cache needs to run after pip is installed
* This is quick follow up fix after apache#8575
ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022
* This is a good practice to save storage space in
  the Docker images being created
* Also sort pip package lists alphabetically
ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022
* The fix to disable cache needs to run after pip is installed
* This is quick follow up fix after apache#8575
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants