cquery's config function doesn't work with Bzlmod #19823

meteorcloudy · 2023-10-16T09:48:04Z

Description of the bug:

When Bzlmod is enabled, //src/test/shell/integration:configured_query_test fails with

** test_config_function ********************************************************
-- Test log: -----------------------------------------------------------
$TEST_TMPDIR defined: output root default is '/b/f/w/_tmp/7db115b6dfb285ceb10442ec6acaf9d3' and max_idle_secs default is '15'.
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Analyzing: target //test_config_function:tool (0 packages loaded, 0 targets configured)
INFO: Analyzed target //test_config_function:tool (0 packages loaded, 1 target configured).
INFO: Found 1 target...
ERROR: Error doing post analysis query: Evaluation failed: No target (in) //test_config_function:tool could be found in the configuration with checksum '50265a7'
INFO: Elapsed time: 0.333s, Critical Path: 0.00s
INFO: 0 processes.
ERROR: Build did NOT complete successfully

see #19785 (comment)

Which category does this issue belong to?

Configurability

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

To reproduce:

Remove disable_bzlmod from src/test/shell/integration/configured_query_test.sh
bazel test //src/test/shell/integration:configured_query_test --test_filter test_config_function

Which operating system are you running Bazel on?

No response

What is the output of `bazel info release`?

No response

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

It's very likely //src/test/java/com/google/devtools/build/lib/runtime:ConfigCommandTest is failing with the same reason when enabling Bzlmod.

The text was updated successfully, but these errors were encountered:

meteorcloudy · 2023-10-16T09:48:47Z

/cc @gregestren

fmeum · 2023-10-16T12:49:23Z

/cc @katre

Bazel tests status: - Bzlmod disabled: - AnalysisTestCase: to be migrated - ConfigurationTestCase: to be migrated - ConfigCommandTest: to be migrated, probably blocked by a bug - Bzlmod enabled: - BuildViewTestCase: migrated at d51144c - Java integration tests migrated at 8d04711 - Shell integration tests migrated at 175a18d (Bzlmod disabled in some tests) - Python integration tests migrated at 50c8375 (Bzlmod disabled in some tests) - BuildIntegrationTestCase: migrated in this change - Other Java unit tests migrated in this change Issues identified: - cc_shared_library doesn't work well with Bzlmod: #19822 - `bazel config` doesn't work well with Bzlmod: #19823 Fixes #18958 Tracking migration of remaining test cases in #19824 RELNOTES[INC]: Bzlmod is enabled by default, please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. Find more details at #18958 PiperOrigin-RevId: 573827480 Change-Id: I097b4bd7caafc996b034284ee688b8f3d2bca1f7

gregestren · 2023-10-16T21:53:27Z

As @fmeum mentioned at #19785 (comment), you can also demonstrate this with $ bazel config. But this specific error is about cquery's config() function, which is technically different. Updated the title accordingly.

gregestren · 2023-10-16T22:39:39Z

@fmeum I'm not recreating the failure you described in #19785 (comment) at the command line.

I can reproduce the test failure.

But a direct i cquery / config invocation with bazel@head with --enable_bzlmod works for me. What are you doing differently?

$ b/bazel cquery "deps(//$pkg:demo)" --enable_bzlmod
<results snipped>

$ b/bazel config  --enable_bzlmod
Available configurations:
0c35974bbc4114fb6b2aaa97afc0fa05325df6498995fc5ab8048c6e904ce99c k8-fastbuild
f3e09891a52a118ada4747b9fed7d209ab6049a7786710934f5f50e1d0f82f2a fastbuild-noconfig
f60fa7f4f050ef3d0bbd3e9a847759d104c0983e4613406726e82cfac3fc0db1 k8-opt-exec-ST-fad1763555eb (exec)
fb7cccd5d9f2742c9ccf42571051eb647e8f01b2cf7be93309af5916222f86e2 k8-fastbuild

fmeum · 2023-10-17T07:15:15Z

With this setup:

# MODULE.bazel (empty)
# WORKSPACE (empty)
# pkg/BUILD
load("//pkg:rule.bzl", "demo_rule")

sh_binary(name = "tool", srcs = ["tool.sh"])

demo_rule(
    name = 'demo',
    binary = ":tool",
)
# pkg/rule.bzl
def _impl(ctx):
    pass

demo_rule = rule(
    implementation = _impl,
    attrs = {
        "binary": attr.label(cfg = "exec"),
    },
)
# pkg/tool.sh
echo "Hello"
exit 0

I get:

$ USE_BAZEL_VERSION=2ddacab80af1dca3c04e74809827c47ec4bc8498 bazel --nohome_rc --nosystem_rc cquery "deps(//pkg:demo)"
//pkg:demo (cb4f3cc)
@local_config_platform//:host (f3e0989)
//pkg:tool (bac8967)
...
$ USE_BAZEL_VERSION=2ddacab80af1dca3c04e74809827c47ec4bc8498 bazel --nohome_rc --nosystem_rc config
Available configurations:
$ USE_BAZEL_VERSION=2ddacab80af1dca3c04e74809827c47ec4bc8498 bazel --nohome_rc --nosystem_rc cquery "deps(//pkg:demo)" --noenable_bzlmod
//pkg:demo (cb4f3cc)
@local_config_platform//:host (f3e0989)
//pkg:tool (bac8967)
...
$ USE_BAZEL_VERSION=2ddacab80af1dca3c04e74809827c47ec4bc8498 bazel --nohome_rc --nosystem_rc config --noenable_bzlmod
Available configurations:
bac89673cb6a316d6814d09db69e2300dd9796229313c36bc7c06680a1c4d8ec k8-opt-exec-ST-e846b08c7501 (exec)
cb4f3ccb516213afb57b51cb2aba480bc44b184b701bcc30e6dd090e8540af2b k8-fastbuild
f3e09891a52a118ada4747b9fed7d209ab6049a7786710934f5f50e1d0f82f2a fastbuild-noconfig

fmeum · 2023-10-17T09:06:48Z

It seems likely that this is also the cause of bazelbuild/rules_go#3597 (comment).

fmeum · 2023-10-17T10:29:30Z

I will submit a fix shortly.

fmeum · 2023-10-17T10:38:46Z

@bazel-io flag

fmeum · 2023-10-17T11:32:55Z

I submitted #19842.

meteorcloudy · 2023-10-17T12:05:58Z

@bazel-io fork 6.4.0

…update Ensure that `BazelLockFileModule` only updates `MODULE.bazel.lock` if the content of the file needs to change. Every such update changes the file's metadata, which results in Skyframe invalidation of, in particular, all configurations. This broke `bazel config`, which uses `MemoizingEvaluator#getDoneValues()` to directly observe Skyframe state. Compared to the original commit bazelbuild@78db9ae, this cherry-pick does not include the change to `bazel config` as it may not be backwards compatible (changes the exit code in certain situations). Fixes bazelbuild#19823 Closes bazelbuild#19842. PiperOrigin-RevId: 574133346 Change-Id: I5886c91fc6b7b938a7dee59ea75aa7b8afb5b161 Fixes bazelbuild#19843

#19848) …update Ensure that `BazelLockFileModule` only updates `MODULE.bazel.lock` if the content of the file needs to change. Every such update changes the file's metadata, which results in Skyframe invalidation of, in particular, all configurations. This broke `bazel config`, which uses `MemoizingEvaluator#getDoneValues()` to directly observe Skyframe state. Compared to the original commit 78db9ae, this cherry-pick does not include the change to `bazel config` as it may not be backwards compatible (changes the exit code in certain situations). Fixes #19823 Closes #19842. PiperOrigin-RevId: 574133346 Change-Id: I5886c91fc6b7b938a7dee59ea75aa7b8afb5b161 Fixes #19843

iancha1992 · 2023-10-17T20:15:19Z

A fix for this issue has been included in Bazel 6.4.0 RC5. Please test out the release candidate and report any issues as soon as possible. Thanks!

iancha1992 · 2023-10-18T18:16:58Z

@bazel-io fork 7.0.0

meteorcloudy added type: bug untriaged labels Oct 16, 2023

meteorcloudy assigned sgowroji, Pavank1992 and iancha1992 Oct 16, 2023

meteorcloudy added the team-Configurability platforms, toolchains, cquery, select(), config transitions label Oct 16, 2023

meteorcloudy mentioned this issue Oct 16, 2023

Enable Bzlmod in all test cases for Bazel #19824

Closed

4 tasks

sgowroji unassigned sgowroji, Pavank1992 and iancha1992 Oct 16, 2023

gregestren self-assigned this Oct 16, 2023

gregestren added P1 I'll work on this now. (Assignee required) and removed untriaged labels Oct 16, 2023

gregestren changed the title ~~bazel config doesn't work with Bzlmod enabled~~ cquery's config function doesn't work with Bzlmod enabled Oct 16, 2023

gregestren changed the title ~~cquery's config function doesn't work with Bzlmod enabled~~ cquery's config function doesn't work with Bzlmod Oct 16, 2023

This was referenced Oct 17, 2023

ERROR FAILED: //tests/core/transition:hermeticity_test with Bazel@HEAD in Downstream CI bazelbuild/rules_go#3597

Closed

Work around Bazel@HEAD bug breaking hermeticity_test bazelbuild/rules_go#3729

Closed

fmeum assigned fmeum and unassigned gregestren Oct 17, 2023

bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Oct 17, 2023

fmeum mentioned this issue Oct 17, 2023

Fix unconditional Skyframe invalidation with --lockfile_mode=update #19842

Closed

bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Oct 17, 2023

bazel-io mentioned this issue Oct 17, 2023

[6.4.0] cquery's config function doesn't work with Bzlmod #19843

Closed

meteorcloudy added this to the 7.0.0 release blockers milestone Oct 17, 2023

copybara-service bot closed this as completed in 78db9ae Oct 17, 2023

fmeum mentioned this issue Oct 17, 2023

[6.4.0] Fix unconditional Skyframe invalidation with --lockfile_mode=update #19848

Merged

iancha1992 removed this from the 7.0.0 release blockers milestone Oct 18, 2023

bazel-io mentioned this issue Oct 18, 2023

[7.0.0] cquery's config function doesn't work with Bzlmod #19879

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cquery's config function doesn't work with Bzlmod #19823

cquery's config function doesn't work with Bzlmod #19823

meteorcloudy commented Oct 16, 2023 •

edited

Loading

meteorcloudy commented Oct 16, 2023

fmeum commented Oct 16, 2023

gregestren commented Oct 16, 2023

gregestren commented Oct 16, 2023

fmeum commented Oct 17, 2023 •

edited

Loading

fmeum commented Oct 17, 2023 •

edited

Loading

fmeum commented Oct 17, 2023

fmeum commented Oct 17, 2023

fmeum commented Oct 17, 2023

meteorcloudy commented Oct 17, 2023

iancha1992 commented Oct 17, 2023

iancha1992 commented Oct 18, 2023

cquery's config function doesn't work with Bzlmod #19823

cquery's config function doesn't work with Bzlmod #19823

Comments

meteorcloudy commented Oct 16, 2023 • edited Loading

Description of the bug:

Which category does this issue belong to?

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Which operating system are you running Bazel on?

What is the output of bazel info release?

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

meteorcloudy commented Oct 16, 2023

fmeum commented Oct 16, 2023

gregestren commented Oct 16, 2023

gregestren commented Oct 16, 2023

fmeum commented Oct 17, 2023 • edited Loading

fmeum commented Oct 17, 2023 • edited Loading

fmeum commented Oct 17, 2023

fmeum commented Oct 17, 2023

fmeum commented Oct 17, 2023

meteorcloudy commented Oct 17, 2023

iancha1992 commented Oct 17, 2023

iancha1992 commented Oct 18, 2023

meteorcloudy commented Oct 16, 2023 •

edited

Loading

What is the output of `bazel info release`?

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

fmeum commented Oct 17, 2023 •

edited

Loading

fmeum commented Oct 17, 2023 •

edited

Loading