Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

General issue to discuss repeated "out of disk space" on Nodes #2510

Closed
andrew-m-leonard opened this issue Mar 15, 2022 · 10 comments
Closed

General issue to discuss repeated "out of disk space" on Nodes #2510

andrew-m-leonard opened this issue Mar 15, 2022 · 10 comments
Assignees
@sxa @Haroon-Khel
Labels
ansible
Milestone
2022-04 (April)

Comments

@andrew-m-leonard
Copy link
Contributor

There seems to be a persistent issue of nodes running out of disk space, issues contributing to this seem to be:

  • Test jobs failing/aborting and skipping the job cleanup, especially the case when large dumps are produced
  • When multiple build git caches build up on a node, including older no longer built versions, you can end up with 6 or so times 5+Gb for each
  • Redundent docker images building up over time taking unnecessary space
@andrew-m-leonard
Copy link
Contributor Author

docker images -a of node docker-packet-ubuntu2004-x64-1, a lot of maybe unused images, taking a lot of space:

aleonard@docker-packet-ubuntu2004-x64-1:~$ docker images -a
REPOSITORY                                    TAG           IMAGE ID       CREATED         SIZE
opensuse/tumbleweed                           latest        bba85c44a790   34 hours ago    102MB
adoptopenjdk/centos6_build_image              latest        1014cbacf545   3 days ago      10.4GB
opensuse/leap                                 15.3          218db86d72f8   4 days ago      109MB
<none>                                        <none>        fa2f8a42c830   10 days ago     1.33GB
openjdk_container-openj9                      latest        38072a36cfef   10 days ago     1.33GB
<none>                                        <none>        78f33d2b2350   10 days ago     1.33GB
<none>                                        <none>        676c30442233   10 days ago     1.33GB
<none>                                        <none>        ab56392cad80   10 days ago     1.33GB
<none>                                        <none>        9b6ca9bca578   10 days ago     1.33GB
<none>                                        <none>        20123a38a19a   10 days ago     1.33GB
<none>                                        <none>        be98199188ee   10 days ago     1.32GB
<none>                                        <none>        4ac76ee5b2fa   10 days ago     1.32GB
<none>                                        <none>        41fd76d9d8d3   10 days ago     1.32GB
<none>                                        <none>        e3f7db2612ff   10 days ago     1.32GB
<none>                                        <none>        fcd2cc4580ad   10 days ago     1.32GB
<none>                                        <none>        4ce80550515b   10 days ago     1.32GB
<none>                                        <none>        3a43550b9382   10 days ago     1.32GB
<none>                                        <none>        463818a306ba   10 days ago     1.32GB
<none>                                        <none>        d6bd73ef7ac6   10 days ago     1.32GB
<none>                                        <none>        9dbabf95a2a9   10 days ago     1.11GB
<none>                                        <none>        9150d3a06563   10 days ago     1.11GB
<none>                                        <none>        fc16f9a4d882   10 days ago     1.11GB
<none>                                        <none>        cba31368d22f   10 days ago     1.11GB
openjdk_container                             latest        7e19c401ce64   10 days ago     1.05GB
<none>                                        <none>        1a487f8f7446   10 days ago     1.05GB
<none>                                        <none>        5381c52d493f   10 days ago     1.05GB
<none>                                        <none>        e4ed1805a1f3   10 days ago     1.05GB
<none>                                        <none>        7982320aa52a   10 days ago     1.05GB
<none>                                        <none>        66bd45956c86   10 days ago     1.05GB
<none>                                        <none>        0f898ff76ca8   10 days ago     1.05GB
<none>                                        <none>        18dba8837944   10 days ago     1.05GB
<none>                                        <none>        3b24a1ca2704   10 days ago     1.05GB
<none>                                        <none>        b75c5bc8f5e0   10 days ago     1.05GB
<none>                                        <none>        01ff82660522   10 days ago     1.05GB
<none>                                        <none>        e83bb827eeb2   10 days ago     1.05GB
<none>                                        <none>        da96f4e337c9   10 days ago     1.04GB
<none>                                        <none>        3702448dbf5e   10 days ago     1.05GB
<none>                                        <none>        376afeb8cc08   10 days ago     1.04GB
<none>                                        <none>        f94884f8b043   10 days ago     1.04GB
<none>                                        <none>        6520d21d1700   10 days ago     833MB
ubuntu                                        20.04         2b4cba85892a   11 days ago     72.8MB
debian                                        buster-slim   09904dab2f41   2 weeks ago     69.3MB
debian                                        buster        739ca6e61c27   2 weeks ago     114MB
rvcross-final3                                latest        35c0ef095434   2 weeks ago     1.99GB
<none>                                        <none>        38005f811011   2 weeks ago     1.99GB
<none>                                        <none>        a39257ea6dbb   2 weeks ago     1.99GB
<none>                                        <none>        75cd0d398867   2 weeks ago     1.99GB
<none>                                        <none>        f332f4829392   2 weeks ago     1.99GB
<none>                                        <none>        09e10c8fc827   2 weeks ago     1.99GB
<none>                                        <none>        9c37c1cad9ac   2 weeks ago     824MB
<none>                                        <none>        9f28ce5eaee3   2 weeks ago     820MB
<none>                                        <none>        887e72bac04d   2 weeks ago     72.8MB
centos6_haroon_test                           latest        109bb491a621   2 weeks ago     732MB
<none>                                        <none>        4c0ea842a3bc   2 weeks ago     732MB
<none>                                        <none>        3732e7438d8b   3 weeks ago     655MB
<none>                                        <none>        8630544bf69f   3 weeks ago     195MB
quay.io/centos/centos                         stream9       21605418e546   3 weeks ago     149MB
quay.io/centos/centos                         stream        6b9f6997daf6   3 weeks ago     376MB
quay.io/centos/centos                         stream8       6b9f6997daf6   3 weeks ago     376MB
adoptium-packages-linux-jdk-suse              latest        928d64c188f8   4 weeks ago     670MB
adoptium-packages-linux-jdk-redhat            latest        b466761d01aa   4 weeks ago     574MB
adoptium-packages-linux-jdk-debian            latest        67d976afa59f   4 weeks ago     525MB
registry.suse.com/suse/sle15                  latest        05dbff30f3ac   5 weeks ago     116MB
<none>                                        <none>        4926879032e3   5 weeks ago     789MB
<none>                                        <none>        9ddeb042920e   5 weeks ago     789MB
<none>                                        <none>        d2ae298d7b12   5 weeks ago     63.2MB
amazonlinux                                   2             eab2a8d1dc55   5 weeks ago     164MB
oraclelinux                                   8             735b286a464d   5 weeks ago     235MB
eclipse-temurin                               11            3202a8748b83   5 weeks ago     464MB
ubuntu                                        hirsute       7cc39f89fa58   5 weeks ago     80MB
ubuntu                                        focal         54c9d81cbb44   5 weeks ago     72.8MB
ubuntu                                        latest        54c9d81cbb44   5 weeks ago     72.8MB
ubuntu                                        18.04         dcf4d4bef137   5 weeks ago     63.2MB
ubuntu                                        bionic        dcf4d4bef137   5 weeks ago     63.2MB
eclipse-temurin                               11-centos7    88fa86b2f4c2   5 weeks ago     563MB
eclipse-temurin                               11-alpine     88d6cacf91e0   5 weeks ago     327MB
redhat/ubi8                                   latest        b81e86a2cb9a   6 weeks ago     216MB
debian                                        stretch       28c1b600f6da   6 weeks ago     101MB
debian                                        bullseye      04fbdaf87a6a   6 weeks ago     124MB
bash                                          latest        53e520fda182   7 weeks ago     13MB
adoptopenjdk/centos6_build_image              <none>        00530c99b03c   8 weeks ago     10.4GB
oraclelinux                                   7             58b88df2dd02   2 months ago    253MB
registry.access.redhat.com/ubi7/ubi           latest        a44a6860246f   2 months ago    206MB
opensuse/leap                                 15.2          9af2c7b5935b   2 months ago    109MB
rockylinux                                    8             210996f98b85   2 months ago    205MB
fedora                                        35            b78af7a83692   3 months ago    153MB
fedora                                        34            e1f83fe2ddbd   3 months ago    178MB
alpine                                        latest        c059bfaa849c   3 months ago    5.59MB
alpine                                        3.14          0a97eee8041e   4 months ago    5.61MB
registry.access.redhat.com/ubi8/ubi           8.4           b1e63aaae5cf   4 months ago    225MB
registry.access.redhat.com/ubi8/ubi-minimal   8.4           e7685639f552   4 months ago    102MB
centos                                        7             eeb6ee3f44bd   6 months ago    204MB
ubuntu                                        16.04         b6f507652425   6 months ago    135MB
debian                                        8             3aaeab7a4777   11 months ago   129MB
<none>                                        <none>        5321f0c54a59   12 months ago   380MB
<none>                                        <none>        6ad628c5df62   12 months ago   380MB
aqa_alp312                                    latest        b99783a82e46   12 months ago   380MB
<none>                                        <none>        ea84bdc73fc0   12 months ago   380MB
<none>                                        <none>        c3c19c186a87   12 months ago   380MB
<none>                                        <none>        1459072e7a07   12 months ago   380MB
<none>                                        <none>        eb1dc1d65f1d   12 months ago   380MB
<none>                                        <none>        e103f20af811   12 months ago   380MB
<none>                                        <none>        4f32f8c9ea93   12 months ago   380MB
<none>                                        <none>        b724f550b58f   12 months ago   380MB
<none>                                        <none>        e7de2ce8d640   12 months ago   380MB
<none>                                        <none>        f24995a7da3b   12 months ago   380MB
<none>                                        <none>        164750607381   12 months ago   338MB
<none>                                        <none>        3d3f7157123f   12 months ago   338MB
<none>                                        <none>        ddf86f869517   12 months ago   338MB
<none>                                        <none>        ce6e11b0f651   12 months ago   338MB
<none>                                        <none>        9f230e9123d4   12 months ago   338MB
<none>                                        <none>        8cf32fde4324   12 months ago   328MB
<none>                                        <none>        1c78911a653a   12 months ago   328MB
<none>                                        <none>        c71009efc9d7   12 months ago   349MB
aqa_alp311                                    latest        21ecc63d021f   12 months ago   349MB
<none>                                        <none>        8a47f9af5e20   12 months ago   349MB
<none>                                        <none>        2415b6dbd242   12 months ago   349MB
<none>                                        <none>        410d0f3ca7fb   12 months ago   349MB
<none>                                        <none>        bcd151194c2d   12 months ago   349MB
<none>                                        <none>        2cc948614417   12 months ago   349MB
<none>                                        <none>        372db67b8178   12 months ago   349MB
<none>                                        <none>        2076a4d45ac9   12 months ago   349MB
<none>                                        <none>        c7677c415e37   12 months ago   349MB
<none>                                        <none>        f6ed348ae5c1   12 months ago   349MB
<none>                                        <none>        8b8155206088   12 months ago   349MB
<none>                                        <none>        1e8d08c2fdd2   12 months ago   307MB
<none>                                        <none>        3a9dcbcc8886   12 months ago   307MB
<none>                                        <none>        d3e8e1c3b0f9   12 months ago   307MB
<none>                                        <none>        7427db03ca6f   12 months ago   307MB
<none>                                        <none>        5eed5efd0d58   12 months ago   307MB
<none>                                        <none>        1b2ad62015fb   12 months ago   297MB
<none>                                        <none>        057d52a210c4   12 months ago   297MB
aqa_f33                                       latest        da681e973efc   12 months ago   1.37GB
<none>                                        <none>        8d4755485024   12 months ago   1.37GB
aqa_u2004                                     latest        3b55632840e1   12 months ago   967MB
<none>                                        <none>        300a22f1ce9e   12 months ago   967MB
aqa_u1804                                     latest        a2a28e81650e   12 months ago   898MB
<none>                                        <none>        0deaf6170bd7   12 months ago   898MB
<none>                                        <none>        5b0bd2c7747a   12 months ago   611MB
<none>                                        <none>        85bd7478753d   12 months ago   611MB
<none>                                        <none>        1ee7008dbad4   12 months ago   611MB
<none>                                        <none>        013f12592436   12 months ago   611MB
<none>                                        <none>        8ffa5234de38   12 months ago   611MB
<none>                                        <none>        a96ae26657bd   12 months ago   610MB
<none>                                        <none>        046046e506d9   12 months ago   610MB
<none>                                        <none>        3014211d5bfa   12 months ago   610MB
<none>                                        <none>        10b9d03d035b   12 months ago   610MB
<none>                                        <none>        0bc78bdcf242   12 months ago   610MB
<none>                                        <none>        e206f30b10f3   12 months ago   571MB
<none>                                        <none>        e4997de8398d   12 months ago   571MB
<none>                                        <none>        b50b9b632e66   12 months ago   570MB
<none>                                        <none>        d50b8f261548   12 months ago   561MB
<none>                                        <none>        b3c9d9235b25   12 months ago   358MB
<none>                                        <none>        8c886b446bdc   12 months ago   255MB
<none>                                        <none>        653aa9fd1506   12 months ago   63.3MB
opensuse/leap                                 15.1          9a7d78396082   13 months ago   106MB
<none>                                        <none>        99c02687eaac   14 months ago   1.2GB
<none>                                        <none>        c8c86050f7b5   14 months ago   1.2GB
<none>                                        <none>        9452f398debb   14 months ago   1.2GB
<none>                                        <none>        b16f5d793dad   14 months ago   1.2GB
<none>                                        <none>        578052988783   14 months ago   1.2GB
<none>                                        <none>        516492c96861   14 months ago   1.2GB
<none>                                        <none>        d66204e6e4f0   14 months ago   1.2GB
<none>                                        <none>        add6b3c003f6   14 months ago   1.2GB
<none>                                        <none>        19e3cc87f676   14 months ago   1.2GB
<none>                                        <none>        e88a0bd51934   14 months ago   1.16GB
<none>                                        <none>        5f6186b1ffe4   14 months ago   1.16GB
<none>                                        <none>        728c80cdd583   14 months ago   1.16GB
<none>                                        <none>        5c9f327150d1   14 months ago   1.15GB
<none>                                        <none>        439045f6c6f9   14 months ago   945MB
<none>                                        <none>        48bdcada66fb   14 months ago   841MB
<none>                                        <none>        24a811aa545b   14 months ago   841MB
<none>                                        <none>        6a7bfc4d5fbb   14 months ago   630MB
<none>                                        <none>        1f0edd220fd7   14 months ago   630MB
<none>                                        <none>        a0874fe4c7b2   14 months ago   630MB
<none>                                        <none>        275bfe271ddc   14 months ago   630MB
<none>                                        <none>        58d98fe048fb   14 months ago   630MB
<none>                                        <none>        6d195e512e34   14 months ago   630MB
<none>                                        <none>        3ac55b91712a   14 months ago   630MB
<none>                                        <none>        0df38572f720   14 months ago   629MB
<none>                                        <none>        a49fb91411fb   14 months ago   629MB
<none>                                        <none>        9c422d7d2071   14 months ago   629MB
<none>                                        <none>        13682a8de69c   14 months ago   590MB
<none>                                        <none>        70d68766f4f6   14 months ago   590MB
<none>                                        <none>        1b374668b9a8   14 months ago   590MB
<none>                                        <none>        aded7ded3618   14 months ago   581MB
<none>                                        <none>        4ed27076fae7   14 months ago   374MB
<none>                                        <none>        be78a027972e   14 months ago   270MB
<none>                                        <none>        72dd606d45c6   14 months ago   72.9MB
<none>                                        <none>        5c534be56eca   15 months ago   5.61MB
<none>                                        <none>        389fef711851   15 months ago   5.58MB
<none>                                        <none>        f643c72bc252   15 months ago   72.9MB
<none>                                        <none>        2c047404e52d   15 months ago   63.3MB
<none>                                        <none>        b3048463dcef   16 months ago   175MB
testcontainers/ryuk                           0.3.0         03cb4444a82d   21 months ago   12MB
centos                                        6.9           2199b8eb8390   3 years ago     195MB

@sophia-guo
Copy link

Test jobs failing/aborting and skipping the job cleanup, especially the case when large dumps are produced

https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/61/console

[](https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/61/console#)[Pipeline] node
Still waiting to schedule task
Waiting for next available executor on ‘[ci.role.test&&hw.arch.x86&&sw.os.linux](https://ci.adoptopenjdk.net/label/ci.role.test&&hw.arch.x86&&sw.os.linux/)’
Running on [test-docker-ubuntu2110-x64-1](https://ci.adoptopenjdk.net/computer/test-docker-ubuntu2110-x64-1/)[](https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/61/console#)[](https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/61/console#)[](https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/61/console#)[](https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/61/console#)[](https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_hs_extended.openjdk_x86-64_linux/61/console#) in /home/jenkins/workspace/Test_openjdk11_hs_extended.openjdk_x86-64_linux
[Pipeline] {
[Pipeline] timeout
Timeout set to expire in 1 hr 0 min
[Pipeline] {
[Pipeline] cleanWs
[Pipeline] }
[Pipeline] // timeout
[Pipeline] echo
Exception: java.nio.file.FileSystemException: /home/jenkins/workspace/Test_openjdk11_hs_extended.openjdk_x86-64_linux: No space left on device
[Pipeline] }
[Pipeline] // node

For this case job has been assigned to the machine test-docker-ubuntu2110-x64-1 and failed when doing cleanWS. (I cannot log in the machine to see if there is files stays in that workspace besides that machine seems working now https://ci.adoptopenjdk.net/view/work-in-progress/job/grinder_sandbox_new/529/console

Based on this case I suspect that deferred wipeout may ask for more space to do the wipeout as it will copy rename workspace directory to a temporary directory name, then start a background task for deleting that temporary directory to short the build time. PR created in aqa-tests to improve this adoptium/aqa-tests#3467

@andrew-m-leonard
Copy link
Contributor Author

@Haroon-Khel @sxa @smlambert
docker-packet-ubuntu2004-amd-1 ran out of disk space this morning.
I've done a bit of pruning of old docker containers and images which freed up 20Gb, but there are quite a few I don't know the status of, some that maybe can be deleted below?

4b42fc368ea6   centos6_haroon_test   "/bin/bash"           2 weeks ago     Up 2 weeks                                                  centos6_haroon_testimage
bf6f0100e541   centos:7              "/bin/bash"           4 weeks ago     Up 4 weeks                                                  sxa_itest_centos7

Here's the full list of images:

aleonard@docker-packet-ubuntu2004-x64-1:~$ docker images 
REPOSITORY                                    TAG           IMAGE ID       CREATED         SIZE
opensuse/tumbleweed                           latest        bba85c44a790   3 days ago      102MB
adoptopenjdk/centos7_build_image              latest        136d28beec45   5 days ago      8.43GB
adoptopenjdk/centos6_build_image              latest        1014cbacf545   5 days ago      10.4GB
opensuse/leap                                 15.3          218db86d72f8   6 days ago      109MB
openjdk_container-openj9                      latest        38072a36cfef   12 days ago     1.33GB
openjdk_container                             latest        7e19c401ce64   12 days ago     1.05GB
ubuntu                                        20.04         2b4cba85892a   13 days ago     72.8MB
debian                                        buster-slim   09904dab2f41   2 weeks ago     69.3MB
debian                                        buster        739ca6e61c27   2 weeks ago     114MB
rvcross-final3                                latest        35c0ef095434   2 weeks ago     1.99GB
centos6_haroon_test                           latest        109bb491a621   3 weeks ago     732MB
quay.io/centos/centos                         stream9       21605418e546   3 weeks ago     149MB
quay.io/centos/centos                         stream        6b9f6997daf6   4 weeks ago     376MB
quay.io/centos/centos                         stream8       6b9f6997daf6   4 weeks ago     376MB
adoptium-packages-linux-jdk-suse              latest        928d64c188f8   4 weeks ago     670MB
adoptium-packages-linux-jdk-redhat            latest        b466761d01aa   4 weeks ago     574MB
adoptium-packages-linux-jdk-debian            latest        67d976afa59f   4 weeks ago     525MB
registry.suse.com/suse/sle15                  latest        05dbff30f3ac   5 weeks ago     116MB
amazonlinux                                   2             eab2a8d1dc55   5 weeks ago     164MB
oraclelinux                                   8             735b286a464d   5 weeks ago     235MB
eclipse-temurin                               11            3202a8748b83   6 weeks ago     464MB
ubuntu                                        hirsute       7cc39f89fa58   6 weeks ago     80MB
ubuntu                                        focal         54c9d81cbb44   6 weeks ago     72.8MB
ubuntu                                        latest        54c9d81cbb44   6 weeks ago     72.8MB
ubuntu                                        18.04         dcf4d4bef137   6 weeks ago     63.2MB
ubuntu                                        bionic        dcf4d4bef137   6 weeks ago     63.2MB
eclipse-temurin                               11-centos7    88fa86b2f4c2   6 weeks ago     563MB
eclipse-temurin                               11-alpine     88d6cacf91e0   6 weeks ago     327MB
redhat/ubi8                                   latest        b81e86a2cb9a   6 weeks ago     216MB
debian                                        stretch       28c1b600f6da   7 weeks ago     101MB
debian                                        bullseye      04fbdaf87a6a   7 weeks ago     124MB
bash                                          latest        53e520fda182   7 weeks ago     13MB
oraclelinux                                   7             58b88df2dd02   2 months ago    253MB
registry.access.redhat.com/ubi7/ubi           latest        a44a6860246f   2 months ago    206MB
opensuse/leap                                 15.2          9af2c7b5935b   2 months ago    109MB
rockylinux                                    8             210996f98b85   3 months ago    205MB
fedora                                        35            b78af7a83692   3 months ago    153MB
fedora                                        34            e1f83fe2ddbd   3 months ago    178MB
alpine                                        latest        c059bfaa849c   3 months ago    5.59MB
alpine                                        3.14          0a97eee8041e   4 months ago    5.61MB
registry.access.redhat.com/ubi8/ubi           8.4           b1e63aaae5cf   4 months ago    225MB
registry.access.redhat.com/ubi8/ubi-minimal   8.4           e7685639f552   4 months ago    102MB
centos                                        7             eeb6ee3f44bd   6 months ago    204MB
ubuntu                                        16.04         b6f507652425   6 months ago    135MB
debian                                        8             3aaeab7a4777   11 months ago   129MB
aqa_alp312                                    latest        b99783a82e46   12 months ago   380MB
aqa_alp311                                    latest        21ecc63d021f   12 months ago   349MB
aqa_f33                                       latest        da681e973efc   12 months ago   1.37GB
aqa_u2004                                     latest        3b55632840e1   12 months ago   967MB
aqa_u1804                                     latest        a2a28e81650e   12 months ago   898MB
opensuse/leap                                 15.1          9a7d78396082   13 months ago   106MB
testcontainers/ryuk                           0.3.0         03cb4444a82d   21 months ago   12MB
centos                                        6.9           2199b8eb8390   3 years ago     195MB

@smlambert
Copy link
Contributor

Unable to run installer pipeline as docker-packet-ubuntu2004-intel-1 is "no space left on device"

See: https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/73/console

@sxa
Copy link
Member

sxa commented Apr 2, 2022

It's sitting at 82Gb free just now which ought to be enough 🤷🏻

@sxa
Copy link
Member

sxa commented Apr 2, 2022

Ah the other system -amd-1 is the one that's out. @Haroon-Khel we really need to get that other disk online

@sxa
Copy link
Member

sxa commented Apr 2, 2022 •
edited
Loading

@smlambert I've removed one particularly large docker container that I was using for some testing (hadn't realised it had got that large!) and am re-running at https://ci.adoptopenjdk.net/view/work-in-progress/job/Sophia-adoptium-packages-linux-pipeline/74/

@Haroon-Khel
Copy link
Contributor

The containers hosted on docker-packet-ubuntu2004-amd-1 have been moved to another disk on that machine. They are now pulling from 256g of available space, about 5x that of before

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Apr 13, 2022 •
edited
Loading

New disk added to docker-packet-ubuntu2004-intel-1. The containers hosted on that machine now have access to 600g+ of disk space. More can be added if we continue to see bottlenecks

@sxa sxa added this to the 2022-04 (April) milestone Apr 14, 2022
@sxa
Copy link
Member

sxa commented Apr 14, 2022

I'm going to close this now as the current issues that were causing problems have been resolved. We do need to ensure that we do appropriate capacity planning when new releases come out and not just assume there will be sufficient space when adding new things to the pipelines.

@sxa sxa closed this as completed Apr 14, 2022
This was referenced Apr 14, 2022
Closed
Closed
@smlambert smlambert mentioned this issue Mar 28, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sxa sxa

@Haroon-Khel Haroon-Khel

Labels
ansible
Projects
None yet
Milestone
2022-04 (April)
Development

No branches or pull requests
5 participants
@smlambert @sxa @sophia-guo @andrew-m-leonard @Haroon-Khel