Include extraction in the retrying for submodule download

[pietroalbini]

The code to download larger submodules previously used retries around the curl invocation to handle network failures, but we saw in recent build failures that failures can also happen during extraction, for example if a response got terminated early.

This commit moves the retry outwards, wrapping the whole download+extraction function in the retrying code. This means, if the extraction fails the tarball will be re-downloaded.

[rustbot]

r? @jyn514

(rustbot has picked a reviewer for you, use r? to override)

[JohnTitor]

@bors r+ p=51
To fix the CI failure

[bors]

📌 Commit 48291f1 has been approved by JohnTitor

It is now in the queue for this repository.

[bors]

🌲 The tree is currently closed for pull requests below priority 50. This pull request will be tested once the tree is reopened.

[bors]

⌛ Testing commit 48291f1 with merge b030ab677278e6cc1600e4cd54a87b55972e0a55...

[bors]

💔 Test failed - checks-actions

[pietroalbini]

@bors retry

[bors]

⌛ Testing commit 48291f1 with merge 7b96e10b46410dd20cb6f8a39754a59ef62aa3f4...

[pietroalbini]

@bors r-

Superseded by #106682

[rust-log-analyzer]

The job dist-mips64el-linux failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

-- Performing Test CXX_SUPPORTS_FDATA_SECTIONS - Success
-- Looking for os_signpost_interval_begin
-- Looking for os_signpost_interval_begin - not found
-- Found Python3: /usr/bin/python3.10 (found suitable version "3.10.6", minimum required is "3.0") found components: Interpreter 
CMake Error: File /checkout/src/llvm-project/llvm/llvm.spec.in does not exist.
CMake Error at CMakeLists.txt:996 (configure_file):
  configure_file Problem configuring file

-- Linker detection: GNU ld
-- Performing Test HAS_WERROR_GLOBAL_CTORS
-- Performing Test HAS_WERROR_GLOBAL_CTORS - Failed
-- Performing Test HAS_WERROR_GLOBAL_CTORS - Failed
-- Looking for __x86_64__
-- Looking for __x86_64__ - found
CMake Error at CMakeLists.txt:1111 (add_subdirectory):
  add_subdirectory given source "utils/TableGen" which is not an existing


CMake Error at cmake/modules/TableGen.cmake:10 (message):
  LLVM_TABLEGEN_EXE not set
  LLVM_TABLEGEN_EXE not set
Call Stack (most recent call first):
  include/llvm/IR/CMakeLists.txt:2 (tablegen)


-- Configuring incomplete, errors occurred!
See also "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/CMakeFiles/CMakeOutput.log".
See also "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/CMakeFiles/CMakeError.log".
command did not execute successfully, got: exit status: 1


build script failed, must exit now', /cargo/registry/src/git.luolix.top-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5
 finished in 6.994 seconds
Build completed unsuccessfully in 0:01:04

[rust-log-analyzer]

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[fee1-dead]

Stuck in queue?

@bors r-

[JohnTitor]

IIRC bors commands on closed PRs don't work, I guess we have to synchronize the queue or reopen this to r- (it needs Pietro to restore the branch).

[fee1-dead]

I've initiated the synchronization

[ehuss]

@fee1-dead, when clicking the synchronize button, you need to go through the queue after it is finished and clean up things. Synchronize is fairly buggy. For example, if a PR was approved, but failed in CI, Synchronize will cause it to be re-approved. I think there are other issues like try builds getting converted to r+ in some cases. I went through and removed some erroneously approved PRs, but I did not check if any PRs got unapproved inadvertently since I didn't have the list of approved PRs before the synchronize.

[fee1-dead]

thanks for the information, and sorry about not following up.. it was my first time doing a synchronize and I did not know the risks. It is probably best to change the confirmation message such that people will understand exactly what would happen after pushing the button.