Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR: An error occurred during the fetch of repository '_main~bazel_build_deps~bootstrap_repo_cache' in CI #21292

Closed
sgowroji opened this issue Feb 12, 2024 · 2 comments · Fixed by bazelbuild/continuous-integration#1878
Assignees
Labels
breakage P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug

Comments

@sgowroji
Copy link
Member

sgowroji commented Feb 12, 2024

CI: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3652#018d9b79-7e09-47a4-be42-d0ebc248311a

Platform: Multiple

Logs:

ERROR: An error occurred during the fetch of repository '_main~bazel_build_deps~bootstrap_repo_cache':
   Traceback (most recent call last):
	File "/Users/buildkite/builds/bk-imacpro-8/bazel-org-repo-root/bazel/distdir.bzl", line 89, column 42, in _repo_cache_tar_impl
		http_artifacts = parse_http_artifacts(ctx, lockfile_path, ctx.attr.repos)
	File "/Users/buildkite/builds/bk-imacpro-8/bazel-org-repo-root/bazel/src/tools/bzlmod/utils.bzl", line 108, column 13, in parse_http_artifacts
		fail("Could not find all required repos, missing: %s" % missing_repos)
Error in fail: Could not find all required repos, missing: ["abseil-cpp~", "apple_support~", "bazel_skylib~", "blake3~", "c-ares~", "grpc~", "protobuf~", "stardoc~", "rules_cc~", "rules_go~", "rules_java~", "rules_jvm_external~", "rules_kotlin~", "rules_graalvm~", "rules_license~", "rules_pkg~", "rules_proto~", "rules_python~", "upb~", "zlib~", "zstd-jni~", "grpc~~grpc_repo_deps_ext~bazel_gazelle", "grpc~~grpc_repo_deps_ext~bazel_skylib", "grpc~~grpc_repo_deps_ext~com_envoyproxy_protoc_gen_validate", "grpc~~grpc_repo_deps_ext~com_github_cncf_udpa", "grpc~~grpc_repo_deps_ext~com_google_googleapis", "grpc~~grpc_repo_deps_ext~envoy_api", "grpc~~grpc_repo_deps_ext~rules_cc", "rules_kotlin~~rules_kotlin_extensions~com_github_jetbrains_kotlin"]

CC Greenteam @mai93

@sgowroji sgowroji added type: bug breakage team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. untriaged labels Feb 12, 2024
@fmeum
Copy link
Collaborator

fmeum commented Feb 12, 2024

@meteorcloudy Could we make it so that Bazel at HEAD builds first update the lockfile before they build Bazel?

meteorcloudy added a commit to bazelbuild/continuous-integration that referenced this issue Feb 12, 2024
…ne (#1878)

Fixes bazelbuild/bazel#21292

After
bazelbuild/bazel@a54a393,
the canonical repo name returned by Bazel@HEAD no longer matches the
name in MODULE.bazel.lock, therefore we need to run `bazel mod deps
--lockfile_mode=update` to re-generate the lockfile before running the
build.

This is done by changing the postsubmi.yml file for Bazel to a custom
version located in the CI repo, which can be updated by running
`./update-bazel-postsubmit.sh` under `./pipelines`.

We may revert this change after we update Bazel to 7.1 for building
Bazel itself.
@meteorcloudy meteorcloudy added P1 I'll work on this now. (Assignee required) and removed untriaged labels Feb 12, 2024
@meteorcloudy meteorcloudy self-assigned this Feb 12, 2024
@meteorcloudy meteorcloudy reopened this Feb 13, 2024
@meteorcloudy
Copy link
Member

This is still failing in downstream:

https://github.com/bazelbuild/bazel/blame/909a628aab265707abde7f2a0824b56cc3bef701/src/tools/bzlmod/utils.bzl#L137-L140

It's because this function actually always return the old canonical repo name (with version), so even if we update the lockfile, it doesn't work. I think we need to detect which format should be used in this function.

copybara-service bot pushed a commit that referenced this issue Feb 14, 2024
Previously, their sources are included as external repos and mapped to third_party,
after this change, their sources are directly included under the third_party directory.

This change is needed because a54a393 broke the mapping mechanism which depends on the canonical repository name.

Fixes //src/test/shell/bazel:bazel_bootstrap_distfile_tar_test with Bazel@HEAD and Bazel@7.1 https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3657#018da594-e743-44da-8f76-782a8e5c86b1

Related #21292

PiperOrigin-RevId: 606960533
Change-Id: Ia4a81d5730e04964bc06c8f8ee2685364ce8623b
meteorcloudy added a commit to meteorcloudy/bazel that referenced this issue Feb 14, 2024
If `get_canonical_repo_name` no longer returns the repo name with version due to containing bazelbuild@a54a393, the `_module_repo_name` should not either.

Fixes: bazelbuild#21292

Closes bazelbuild#21324.

PiperOrigin-RevId: 606646238
Change-Id: I8835a84842c2c66929586b39156eb9f5a541652f
meteorcloudy added a commit to meteorcloudy/bazel that referenced this issue Feb 14, 2024
Previously, their sources are included as external repos and mapped to third_party,
after this change, their sources are directly included under the third_party directory.

This change is needed because bazelbuild@a54a393 broke the mapping mechanism which depends on the canonical repository name.

Fixes //src/test/shell/bazel:bazel_bootstrap_distfile_tar_test with Bazel@HEAD and Bazel@7.1 https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3657#018da594-e743-44da-8f76-782a8e5c86b1

Related bazelbuild#21292

PiperOrigin-RevId: 606960533
Change-Id: Ia4a81d5730e04964bc06c8f8ee2685364ce8623b
github-merge-queue bot pushed a commit that referenced this issue Feb 14, 2024
…21349)

Fixes //src/test/py/bazel:bazel_vendor_test in ipv6-only environment …
…on macOS

https://buildkite.com/bazel/bazel-bazel-macos-ninja/builds/514

PiperOrigin-RevId: 606599015
Change-Id: I36a8534d6676bc5c3c9a0157eea28fb033e9cf3e

-------

Fixes //src/test/shell/bazel:bazel_test_test in ipv6-only environment…
… on macOS

https://buildkite.com/bazel/bazel-bazel-macos-ninja/builds/517

PiperOrigin-RevId: 606603953
Change-Id: Id730a0457e2a6bc1ac5371cbbce25c4acd25ab9d

-----

Fixes _module_repo_name when building with Bazel@HEAD or Bazel 7.1
If `get_canonical_repo_name` no longer returns the repo name with
version due to containing a54a393, the `_module_repo_name` should not
either.

Fixes: #21292

Closes #21324.

PiperOrigin-RevId: 606646238
Change-Id: I8835a84842c2c66929586b39156eb9f5a541652f

-----

Make sure generate_dist_lockfile works in ipv6-only environment
Fixes
https://buildkite.com/bazel/bazel-bazel-macos-ninja/builds/534#018da46c-5aff-45ea-8b66-937e676b09b2

PiperOrigin-RevId: 606951013
Change-Id: I9336c9b8a173ed464b0fd3dab22a5dc1614b4c62

----

Fix googleapis and remoteapis sources in bootstrap distfile
Previously, their sources are included as external repos and mapped to
third_party,
after this change, their sources are directly included under the
third_party directory.

This change is needed because
a54a393
broke the mapping mechanism which depends on the canonical repository
name.

Fixes //src/test/shell/bazel:bazel_bootstrap_distfile_tar_test with
Bazel@HEAD and Bazel@7.1
https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3657#018da594-e743-44da-8f76-782a8e5c86b1

Related #21292

PiperOrigin-RevId: 606960533
Change-Id: Ia4a81d5730e04964bc06c8f8ee2685364ce8623b
fweikert pushed a commit to fweikert/bazel that referenced this issue Feb 23, 2024
If `get_canonical_repo_name` no longer returns the repo name with version due to containing bazelbuild@a54a393, the `_module_repo_name` should not either.

Fixes: bazelbuild#21292

Closes bazelbuild#21324.

PiperOrigin-RevId: 606646238
Change-Id: I8835a84842c2c66929586b39156eb9f5a541652f
fweikert added a commit that referenced this issue Feb 23, 2024
…21486)

If `get_canonical_repo_name` no longer returns the repo name with
version due to containing
a54a393,
the `_module_repo_name` should not either.

Fixes: #21292

Closes #21324.

PiperOrigin-RevId: 606646238
Change-Id: I8835a84842c2c66929586b39156eb9f5a541652f

Co-authored-by: Yun Peng <pcloudy@google.com>
fweikert added a commit to fweikert/bazel that referenced this issue Mar 5, 2024
…azelbuild#21486)

If `get_canonical_repo_name` no longer returns the repo name with
version due to containing
bazelbuild@a54a393,
the `_module_repo_name` should not either.

Fixes: bazelbuild#21292

Closes bazelbuild#21324.

PiperOrigin-RevId: 606646238
Change-Id: I8835a84842c2c66929586b39156eb9f5a541652f

Co-authored-by: Yun Peng <pcloudy@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breakage P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug
Projects
None yet
3 participants