[SPARK-48029][INFRA] Update the packages name removed in building the spark docker image #46258
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
panbingkun
changed the title
panbingkun
changed the title
HyukjinKwon
approved these changes
Apr 29, 2024
dongjoon-hyun
approved these changes
Apr 29, 2024
|
Merged to master. |
JacobZheng0927
pushed a commit
to JacobZheng0927/spark
that referenced
this pull request
May 11, 2024
… spark docker image ### What changes were proposed in this pull request? The pr aims to update the packages name removed in building the spark docker image. ### Why are the changes needed? When our default image base was switched from `ubuntu 20.04` to `ubuntu 22.04`, the unused installation package in the base image has changed, in order to eliminate some warnings in building images and free disk space more accurately, we need to correct it. Before: ``` apache#35 [29/31] RUN apt-get remove --purge -y '^aspnet.*' '^dotnet-.*' '^llvm-.*' 'php.*' '^mongodb-.*' snapd google-chrome-stable microsoft-edge-stable firefox azure-cli google-cloud-sdk mono-devel powershell libgl1-mesa-dri || true apache#35 0.489 Reading package lists... apache#35 0.505 Building dependency tree... apache#35 0.507 Reading state information... apache#35 0.511 E: Unable to locate package ^aspnet.* apache#35 0.511 E: Couldn't find any package by glob '^aspnet.*' apache#35 0.511 E: Couldn't find any package by regex '^aspnet.*' apache#35 0.511 E: Unable to locate package ^dotnet-.* apache#35 0.511 E: Couldn't find any package by glob '^dotnet-.*' apache#35 0.511 E: Couldn't find any package by regex '^dotnet-.*' apache#35 0.511 E: Unable to locate package ^llvm-.* apache#35 0.511 E: Couldn't find any package by glob '^llvm-.*' apache#35 0.511 E: Couldn't find any package by regex '^llvm-.*' apache#35 0.511 E: Unable to locate package ^mongodb-.* apache#35 0.511 E: Couldn't find any package by glob '^mongodb-.*' apache#35 0.511 EPackage 'php-crypt-gpg' is not installed, so not removed apache#35 0.511 Package 'php' is not installed, so not removed apache#35 0.511 : Couldn't find any package by regex '^mongodb-.*' apache#35 0.511 E: Unable to locate package snapd apache#35 0.511 E: Unable to locate package google-chrome-stable apache#35 0.511 E: Unable to locate package microsoft-edge-stable apache#35 0.511 E: Unable to locate package firefox apache#35 0.511 E: Unable to locate package azure-cli apache#35 0.511 E: Unable to locate package google-cloud-sdk apache#35 0.511 E: Unable to locate package mono-devel apache#35 0.511 E: Unable to locate package powershell apache#35 DONE 0.5s apache#36 [30/31] RUN apt-get autoremove --purge -y apache#36 0.063 Reading package lists... apache#36 0.079 Building dependency tree... apache#36 0.082 Reading state information... apache#36 0.088 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. apache#36 DONE 0.4s ``` After: ``` apache#38 [32/36] RUN apt-get remove --purge -y 'gfortran-11' 'humanity-icon-theme' 'nodejs-doc' || true apache#38 0.066 Reading package lists... apache#38 0.087 Building dependency tree... apache#38 0.089 Reading state information... apache#38 0.094 The following packages were automatically installed and are no longer required: apache#38 0.094 at-spi2-core bzip2-doc dbus-user-session dconf-gsettings-backend apache#38 0.095 dconf-service gsettings-desktop-schemas gtk-update-icon-cache apache#38 0.095 hicolor-icon-theme libatk-bridge2.0-0 libatk1.0-0 libatk1.0-data apache#38 0.095 libatspi2.0-0 libbz2-dev libcairo-gobject2 libcolord2 libdconf1 libepoxy0 apache#38 0.095 libgfortran-11-dev libgtk-3-common libjs-highlight.js libllvm11 apache#38 0.095 libncurses-dev libncurses5-dev libphobos2-ldc-shared98 libreadline-dev apache#38 0.095 librsvg2-2 librsvg2-common libvte-2.91-common libwayland-client0 apache#38 0.095 libwayland-cursor0 libwayland-egl1 libxdamage1 libxkbcommon0 apache#38 0.095 session-migration tilix-common xkb-data apache#38 0.095 Use 'apt autoremove' to remove them. apache#38 0.096 The following packages will be REMOVED: apache#38 0.096 adwaita-icon-theme* gfortran* gfortran-11* humanity-icon-theme* libgtk-3-0* apache#38 0.096 libgtk-3-bin* libgtkd-3-0* libvte-2.91-0* libvted-3-0* nodejs-doc* apache#38 0.096 r-base-dev* tilix* ubuntu-mono* apache#38 0.248 0 upgraded, 0 newly installed, 13 to remove and 0 not upgraded. apache#38 0.248 After this operation, 99.6 MB disk space will be freed. ... (Reading database ... 70597 files and directories currently installed.) apache#38 0.304 Removing r-base-dev (4.1.2-1ubuntu2) ... apache#38 0.319 Removing gfortran (4:11.2.0-1ubuntu1) ... apache#38 0.340 Removing gfortran-11 (11.4.0-1ubuntu1~22.04) ... apache#38 0.356 Removing tilix (1.9.4-2build1) ... apache#38 0.377 Removing libvted-3-0:amd64 (3.10.0-1ubuntu1) ... apache#38 0.392 Removing libvte-2.91-0:amd64 (0.68.0-1) ... apache#38 0.407 Removing libgtk-3-bin (3.24.33-1ubuntu2) ... apache#38 0.422 Removing libgtkd-3-0:amd64 (3.10.0-1ubuntu1) ... apache#38 0.436 Removing nodejs-doc (12.22.9~dfsg-1ubuntu3.4) ... apache#38 0.457 Removing libgtk-3-0:amd64 (3.24.33-1ubuntu2) ... apache#38 0.488 Removing ubuntu-mono (20.10-0ubuntu2) ... apache#38 0.754 Removing humanity-icon-theme (0.6.16) ... apache#38 1.362 Removing adwaita-icon-theme (41.0-1ubuntu1) ... apache#38 1.537 Processing triggers for libc-bin (2.35-0ubuntu3.6) ... apache#38 1.566 Processing triggers for mailcap (3.70+nmu1ubuntu1) ... apache#38 1.577 Processing triggers for libglib2.0-0:amd64 (2.72.4-0ubuntu2.2) ... (Reading database ... 56946 files and directories currently installed.) apache#38 1.645 Purging configuration files for libgtk-3-0:amd64 (3.24.33-1ubuntu2) ... apache#38 1.657 Purging configuration files for ubuntu-mono (20.10-0ubuntu2) ... apache#38 1.670 Purging configuration files for humanity-icon-theme (0.6.16) ... apache#38 1.682 Purging configuration files for adwaita-icon-theme (41.0-1ubuntu1) ... apache#38 DONE 1.7s apache#39 [33/36] RUN apt-get autoremove --purge -y apache#39 0.061 Reading package lists... apache#39 0.075 Building dependency tree... apache#39 0.077 Reading state information... apache#39 0.083 The following packages will be REMOVED: apache#39 0.083 at-spi2-core* bzip2-doc* dbus-user-session* dconf-gsettings-backend* apache#39 0.083 dconf-service* gsettings-desktop-schemas* gtk-update-icon-cache* apache#39 0.083 hicolor-icon-theme* libatk-bridge2.0-0* libatk1.0-0* libatk1.0-data* apache#39 0.083 libatspi2.0-0* libbz2-dev* libcairo-gobject2* libcolord2* libdconf1* apache#39 0.083 libepoxy0* libgfortran-11-dev* libgtk-3-common* libjs-highlight.js* apache#39 0.083 libllvm11* libncurses-dev* libncurses5-dev* libphobos2-ldc-shared98* apache#39 0.083 libreadline-dev* librsvg2-2* librsvg2-common* libvte-2.91-common* apache#39 0.083 libwayland-client0* libwayland-cursor0* libwayland-egl1* libxdamage1* apache#39 0.083 libxkbcommon0* session-migration* tilix-common* xkb-data* apache#39 0.231 0 upgraded, 0 newly installed, 36 to remove and 0 not upgraded. apache#39 0.231 After this operation, 124 MB disk space will be freed. ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - Manually test. - Pass GA. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#46258 from panbingkun/remove_packages_on_ubuntu. Authored-by: panbingkun <panbingkun@baidu.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The pr aims to update the packages name removed in building the spark docker image.
Why are the changes needed?
When our default image base was switched from
ubuntu 20.04toubuntu 22.04, the unused installation package in the base image has changed, in order to eliminate some warnings in building images and free disk space more accurately, we need to correct it.Before:
After:
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Was this patch authored or co-authored using generative AI tooling?
No.