Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel query --output=location does not print source file paths #8900

Closed
wchargin opened this issue Jul 15, 2019 · 5 comments
Closed

bazel query --output=location does not print source file paths #8900

wchargin opened this issue Jul 15, 2019 · 5 comments
Assignees
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Performance Issues for Performance teams

Comments

@wchargin
Copy link
Contributor

Description of the problem / feature request:

The docs for --output=location say:

For source files, the location of line 1 of the actual file is printed.

This is what I want, but it is not what happens: instead, what is
printed is the location of a reference to the source file from a BUILD
file. There appears to be no way to actually turn a label for a source
file into that source file’s path on disk.

Feature requests: what underlying problem are you trying to solve with this feature?

N/A

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

$ cd "$(mktemp -d)"
$ >WORKSPACE
$ >BUILD <<EOF cat
> py_binary(
>   name = "main",
>   srcs = ["main.py"],
> )
> EOF
$ >main.py
$ bazel query //:main.py --output=location 2>/dev/null
/tmp/tmp.97B2nY3jsI/BUILD:3:10: source file //:main.py

Not even --output=proto includes the promised information:

$ bazel query //:main.py --output=proto | strings
//:main.py
/tmp/tmp.97B2nY3jsI/BUILD:3:10*

What operating system are you running Bazel on?

gLinux (like Debian)

What's the output of bazel info release?

release 0.28.0

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

N/A

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

Nope.

Any other information, logs, or outputs that you want to share?

Nope.

@iirina iirina added team-Bazel General Bazel product/strategy issues untriaged labels Jul 16, 2019
@dslomov dslomov added team-Core Skyframe, bazel query, BEP, options parsing, bazelrc and removed team-Bazel General Bazel product/strategy issues labels Jul 19, 2019
@shreyax shreyax self-assigned this Apr 20, 2020
@janakdr janakdr removed the untriaged label Apr 20, 2020
@jin
Copy link
Member

jin commented May 12, 2020

@janakdr could you assign a priority level for triaged issues, please?

@janakdr janakdr added the P3 We're not considering working on this, but happy to review a PR. (No assignee) label May 12, 2020
@meisterT meisterT assigned zhengwei143 and unassigned shreyax Oct 13, 2020
@meisterT meisterT added team-Performance Issues for Performance teams and removed team-Core Skyframe, bazel query, BEP, options parsing, bazelrc labels Oct 13, 2020
bazel-io pushed a commit that referenced this issue Oct 21, 2020
…t=location `instead of targets.

Added an additional flag `[no]incompatible_display_source_file_location` to override the default behaviour (when false) to print the source file targets instead of their locations. Additionally, the  `relative_locations` flag will similarly display the relative location of the source file.

This #12322 describes the issue and potential migration recipes.

Example of new configurations for blaze query location=output with `incompatible_display_source_file_location` and `relative_locations` flags below:

1. Default behaviour: Displays absolute paths to both BUILD and source file
```
$ bazel query blazetest:main.py --output=location --incompatible_display_source_file_location
/home/tanzhengwei/blazetest/BUILD:1:10: source file /home/tanzhengwei/blazetest/main.py:1:1
```

2. Displays relative paths to both BUILD and source file
```
$ bazel query blazetest:main.py --output=location --relative_locations --incompatible_display_source_file_location
blazetest/BUILD:1:10: source file blazetest/main.py:1:1
```

3. Displays relative path to BUILD file and source file target
```
$ bazel query blazetest:main.py --output=location --relative_locations --noincompatible_display_source_file_location
blazetest/BUILD:1:10: source file //blazetest:main.py
```

4. Displays absolute path to BUILD file and source file target
```
$ bazel query blazetest:main.py --output=location --noincompatible_display_source_file_location
/home/tanzhengwei/blazetest/BUILD:1:10: source file //blazetest:main.py
```

RELNOTES: Added flag `incompatible_display_source_file_location` for `blaze query location=output` to print the location of line 1 of the actual source files instead of the source file targets. Provides a solution to #8900.
PiperOrigin-RevId: 338240601
bazel-io pushed a commit that referenced this issue Oct 23, 2020
This PR is a patch to fix this [commit](f00f911) after being brought up [here](#12322 (comment)). This aims to provide a fix for #8900.

The goal of this PR is to align the output of `--output=location` with its current definition in the [documentation](https://docs.bazel.build/versions/master/query.html#output-location). With the flag `--output=location`, source files are supposed to output "the location of line 1 of the actual file" but only displayed its location in the BUILD file prior to the commit.

**Expected behavior by definition:**
```
bazel query bazeltest:main.py --output=location
/home/tanzhengwei/bazeltest/main.py:1:1: source file //bazeltest:main.py
```

The previous [commit](f00f911) wrongly modified the output to the following (with the `--incompatible_display_source_file_location` flag):
```
bazel query bazeltest:main.py --output=location --incompatible_display_source_file_location
/home/tanzhengwei/bazeltest/BUILD:1:10: source file /home/tanzhengwei/bazeltest/main.py:1:1
```

This PR fixes it to output to the correct implementation (with the `--incompatible_display_source_file_location` flag):
```
bazel query bazeltest:main.py --output=location --incompatible_display_source_file_location
/home/tanzhengwei/bazeltest/main.py:1:1: source file //bazeltest:main.py
```

Closes #12329.

PiperOrigin-RevId: 338620611
@zhengwei143
Copy link
Contributor

This issue will be resolved with #12322.

@zhengwei143
Copy link
Contributor

We are re-thinking the changes that fixed this issue and are considering changing the documentation to match this behaviour instead.

But first we would like to check whether this would be a big issue with you (that it will continue to print the location of the source file within the BUILD file instead)?

@wchargin
Copy link
Contributor Author

wchargin commented Dec 7, 2020

It would certainly be inconvenient. This issue has 6 upvotes (not
including me), which puts it in the top 3.5% of open issues sorted by
vote count.

Finding the source file on disk is often useful, but I’ve never wanted
this to produce a location in a BUILD file (the bad behavior).
If I wanted that, I’d look at same_pkg_direct_rdeps or something.

@zhengwei143
Copy link
Contributor

Thanks for the input, we have also decided internally to carry on with this change.

bazel-io pushed a commit that referenced this issue Dec 10, 2020
…location output formatters.

In order to align proto and xml output formatters with the fix for #8900 (where the location of source files should be its actual location but not its location in the BUILD file).

PiperOrigin-RevId: 346743348
bazel-io pushed a commit that referenced this issue Jan 21, 2021
Baseline: 37a429a

Cherry picks:

   + a689d67:
     Use getRunfilesPath for run_under executable path generation.
     getRootRelativePath doesn't return a valid runfiles path for
     external source files anymore after the recent external source
     root change. Also, it won't work for external labels either once
     the --nolegacy_external_runfiles becomes default. This fixes
     issue #12545.
   + d90ec67:
     Fix NPE when coveragerunner is not set on the toolchain.
   + 8555789:
     Fix the classic query package-loading cutoff optimization with
     external workspaces.
   + d113d74:
     Update turbine
   + 1489f0f:
     Support Scala3 .tasty files
   + 0d2d95c:
     Update to java_tools javac11 release 10.5 (#12647)
   + a9419f3:
     Fix common prefix for instrumentation filter
   + 84fadcf:
     Fix builds for filegroup targets with incompatible dependencies
   + e43825d:
     Revert "Remove
     --incompatible_blacklisted_protos_requires_proto_info"
   + 082d58d:
     Transform roots along with paths during output deletion.
   + e8835c1:
     AttributeContainer.Large now handles more than 127 attributes.
   + e1e8734:
     Add an env attribute to all test and binary rule classes
   + a87d7ed:
     Take no action to prefetch empty artifacts.
   + 3e969ff:
     Fix a couple of bugs with Incompatible Target Skipping
   + e667082:
     Pass --host_action_env to host options hostActionEnvironment
     attribute
   + 07400c0:
     Add --{no,}autodetect_server_javabase.
   + c833660:
     Only treat "env" and "env_inherit" attrs specially for native
     rules
   + 6a60b30:
     Fix coverage support when using default_java_toolchain. (#12801)
   + 4158a6f:
     Revert JacocoCoverage target to remote_java_tools_java_import
     and add a new target for remore_java_tools_filegroup. (#12813)
   + f6d30cf:
     Add windows_msvc back to conditions in bazel_tools.

New features:

  - Starlark-defined flags can now be shorthanded using --flag_alias.

Important changes:

  - Add --starlark:file option. This adds a capability to the
    (cquery)[https://docs.bazel.build/versions/master/cquery.html]
    feature in `--output=starlark` mode so that the expression to
    format output may
    be specified in a file.
  - Error messages emitted when an action fails are reworked to be
    more informative about the failing action. Some tooling may have
    to be updated as a result.
  - Querying with output=location now allows the relative_locations
    flag to properly display relative locations instead of the full
    path. Fixes #3497.
  - --flag_alias can now be used without
    --experimental_enable_flag_alias
  - Remove no-op `--deep_execroot` flag
  - The BEP uses `AbortReason.OUT_OF_MEMORY` for abort events when
    the build tool is crashing due to OOM.
  - Added flag `incompatible_display_source_file_location` for `blaze
    query location=output` to print the location of line 1 of the
    actual source files instead of the source file targets. Provides
    a solution to #8900.
  - The Starlark json module is now available.
    Use json.encode(x) to encode a Starlark value as JSON.
    struct.to_json(x) is deprecated and will be disabled by
    the --incompatible_struct_has_no_methods flag.
  - The flag `--incompatible_objc_compile_info_migration` is enabled
    by default.  See #10854.
  - The flag `--incompatible_objc_provider_remove_compile_info` is
    enabled by default.  See #11359.
  - Add `relative_ast_path` feature for darwin builds to relativize
    swiftmodule paths for debugging
  - Use proto.encode_text(x) to encode a Starlark value as textproto.
    struct.to_proto() is deprecated and will be disabled by
    the --incompatible_struct_has_no_methods flag.
    Both functions now reject list/dict fields that contain list/dict
    elements.
  - Add --starlark:file option. This adds a capability to the
    (cquery)[https://docs.bazel.build/versions/master/cquery.html]
    feature in `--output=starlark` mode so that the expression to
    format output may be specified in a file.
    See [Configured Query Starlark
    Output](//docs.google.com/document/d/1kL6Tdmp6uLBa9lq_DbUSjIC87glO
    zKIyPoBeF95Rs4c/edit)
  - Flipped --incompatble_proto_output_v2 for aquery.
  - The --incompatible_load_java_rules_from_bzl flag is now a no-op.
  - The --incompatible_load_proto_rules_from_bzl flag is now a no-op.
  - Flipped --incompatible_force_strict_header_check_from_starlark
  - --incompatible_string_replace_count is flipped and removed
    (#11244)
  - Bazel skips incompatible targets based on target platform
    and `target_compatible_with` contents. See
    https://docs.bazel.build/versions/master/platforms.html for more
    details.
  - Bazel returns exit code 36 (rather than 1) if it fails to start a
    subprocess in a local sandbox due to environmental issues, for
    example, if the argument list is too long.
  - //tools/build_defs/pkg:pkg_rpm is no longer built in to Bazel.
    See #11218 for
    instructions
    on how to migrate to the replacement.
  - Javac now supports multiplex workers.
  - The `--default_ios_provisioning_profile` flag has been removed
    (it was a no-op).
  - Add support for using AndroidX dependencies in
    data-binding-enabled targets.
  - Fix data-binding generation for android_local_test.
  - Enable debug_prefix_map_pwd_is_dot feature by default on macOS,
    this passes `-fdebug-prefix-map=$PWD=.` for every compile to
    remove absolute paths from debug info.
  - --incompatible_run_shell_command_string is enabled by default
    (#5903)
  - py_binary now tolerates package paths that contain hyphens ('-').
    Note that such paths might not be importable from within Python
    code.
  - C++ Starlark API requires linker_inputs wrapping library_to_link.
    #10860
  - Toolchain rule is extended with target_settings attribute.
  - --incompatible_restrict_string_escapes=true is now the default.
    Unnecessary backslashes such as "\." in string literals are now
    an error, instead of being silently treated as "\\.".
    To fix the error while preserving behavior, double the backlash.
    However, the error is often a sign that the original code was
    wrong.
  - Propagate instrumented files for transitive sources of
    `android_library` and `android_binary`
  - --local_resources and --incompatible_remove_local_resources have
    been removed. If you've been setting --local resources or
    --incompatible_remove_local_resources=false, you must migrate to
    using --local_ram_resources and --local_cpu_resources instead.
  - Update rules_cc to commit b1c40e1de81913a3c40e5948f78719c28152486d
  - --incompatible_avoid_conflict_dlls=true is now the default.
  - Dynamic execution now uses the new scheduler by default.
  - Dynamic execution now uses the new scheduler by default.
  - The new dynamic scheduler is now the default.

This release contains contributions from many people at Google, as well as Adam Liddell, Akira Baruah, Alexander Grund, Alex Eagle, Andrew Z Allen, Austin Schuh, Benjamin Peterson, Benson Muite, Brentley Jones, Cristian Hancila, Dan Halperin, Daniel Wagner-Hall, Dmitry Ivankov, Dmitry Ivankov, erenon, Eric Cousineau, Greg Estren, Gregor Jasny, Grzegorz Lukasik, Grzegorz Lukasik, hollste, Joe Lencioni, johnjbarton, Jonathan Perry, Jonathon Belotti, Keith Smiley, Kevin Gessner, Matt Davis, Matt Mackay, Menny Even Danan, Neeraj Gupta, Philipp Schrader, Ricardo Delfin, Ryan Beasley, Samuel Giddins, Simon Bjorklen, Simon Stewart, Stiopa Koltsov, Thi Doan, ThomasCJY, Timothy Klim, Tom de Goede, vectoralpha, V Vn Ngha, William A Rowe Jr, Xavier Bonaventura, Yannic Bonenberger, Yannic.
hrfuller pushed a commit to hrfuller/bazel that referenced this issue Jan 26, 2021
Baseline: 37a429a

Cherry picks:

   + a689d67:
     Use getRunfilesPath for run_under executable path generation.
     getRootRelativePath doesn't return a valid runfiles path for
     external source files anymore after the recent external source
     root change. Also, it won't work for external labels either once
     the --nolegacy_external_runfiles becomes default. This fixes
     issue bazelbuild#12545.
   + d90ec67:
     Fix NPE when coveragerunner is not set on the toolchain.
   + 8555789:
     Fix the classic query package-loading cutoff optimization with
     external workspaces.
   + d113d74:
     Update turbine
   + 1489f0f:
     Support Scala3 .tasty files
   + 0d2d95c:
     Update to java_tools javac11 release 10.5 (bazelbuild#12647)
   + a9419f3:
     Fix common prefix for instrumentation filter
   + 84fadcf:
     Fix builds for filegroup targets with incompatible dependencies
   + e43825d:
     Revert "Remove
     --incompatible_blacklisted_protos_requires_proto_info"
   + 082d58d:
     Transform roots along with paths during output deletion.
   + e8835c1:
     AttributeContainer.Large now handles more than 127 attributes.
   + e1e8734:
     Add an env attribute to all test and binary rule classes
   + a87d7ed:
     Take no action to prefetch empty artifacts.
   + 3e969ff:
     Fix a couple of bugs with Incompatible Target Skipping
   + e667082:
     Pass --host_action_env to host options hostActionEnvironment
     attribute
   + 07400c0:
     Add --{no,}autodetect_server_javabase.
   + c833660:
     Only treat "env" and "env_inherit" attrs specially for native
     rules
   + 6a60b30:
     Fix coverage support when using default_java_toolchain. (bazelbuild#12801)
   + 4158a6f:
     Revert JacocoCoverage target to remote_java_tools_java_import
     and add a new target for remore_java_tools_filegroup. (bazelbuild#12813)
   + f6d30cf:
     Add windows_msvc back to conditions in bazel_tools.

New features:

  - Starlark-defined flags can now be shorthanded using --flag_alias.

Important changes:

  - Add --starlark:file option. This adds a capability to the
    (cquery)[https://docs.bazel.build/versions/master/cquery.html]
    feature in `--output=starlark` mode so that the expression to
    format output may
    be specified in a file.
  - Error messages emitted when an action fails are reworked to be
    more informative about the failing action. Some tooling may have
    to be updated as a result.
  - Querying with output=location now allows the relative_locations
    flag to properly display relative locations instead of the full
    path. Fixes bazelbuild#3497.
  - --flag_alias can now be used without
    --experimental_enable_flag_alias
  - Remove no-op `--deep_execroot` flag
  - The BEP uses `AbortReason.OUT_OF_MEMORY` for abort events when
    the build tool is crashing due to OOM.
  - Added flag `incompatible_display_source_file_location` for `blaze
    query location=output` to print the location of line 1 of the
    actual source files instead of the source file targets. Provides
    a solution to bazelbuild#8900.
  - The Starlark json module is now available.
    Use json.encode(x) to encode a Starlark value as JSON.
    struct.to_json(x) is deprecated and will be disabled by
    the --incompatible_struct_has_no_methods flag.
  - The flag `--incompatible_objc_compile_info_migration` is enabled
    by default.  See bazelbuild#10854.
  - The flag `--incompatible_objc_provider_remove_compile_info` is
    enabled by default.  See bazelbuild#11359.
  - Add `relative_ast_path` feature for darwin builds to relativize
    swiftmodule paths for debugging
  - Use proto.encode_text(x) to encode a Starlark value as textproto.
    struct.to_proto() is deprecated and will be disabled by
    the --incompatible_struct_has_no_methods flag.
    Both functions now reject list/dict fields that contain list/dict
    elements.
  - Add --starlark:file option. This adds a capability to the
    (cquery)[https://docs.bazel.build/versions/master/cquery.html]
    feature in `--output=starlark` mode so that the expression to
    format output may be specified in a file.
    See [Configured Query Starlark
    Output](//docs.google.com/document/d/1kL6Tdmp6uLBa9lq_DbUSjIC87glO
    zKIyPoBeF95Rs4c/edit)
  - Flipped --incompatble_proto_output_v2 for aquery.
  - The --incompatible_load_java_rules_from_bzl flag is now a no-op.
  - The --incompatible_load_proto_rules_from_bzl flag is now a no-op.
  - Flipped --incompatible_force_strict_header_check_from_starlark
  - --incompatible_string_replace_count is flipped and removed
    (bazelbuild#11244)
  - Bazel skips incompatible targets based on target platform
    and `target_compatible_with` contents. See
    https://docs.bazel.build/versions/master/platforms.html for more
    details.
  - Bazel returns exit code 36 (rather than 1) if it fails to start a
    subprocess in a local sandbox due to environmental issues, for
    example, if the argument list is too long.
  - //tools/build_defs/pkg:pkg_rpm is no longer built in to Bazel.
    See bazelbuild#11218 for
    instructions
    on how to migrate to the replacement.
  - Javac now supports multiplex workers.
  - The `--default_ios_provisioning_profile` flag has been removed
    (it was a no-op).
  - Add support for using AndroidX dependencies in
    data-binding-enabled targets.
  - Fix data-binding generation for android_local_test.
  - Enable debug_prefix_map_pwd_is_dot feature by default on macOS,
    this passes `-fdebug-prefix-map=$PWD=.` for every compile to
    remove absolute paths from debug info.
  - --incompatible_run_shell_command_string is enabled by default
    (bazelbuild#5903)
  - py_binary now tolerates package paths that contain hyphens ('-').
    Note that such paths might not be importable from within Python
    code.
  - C++ Starlark API requires linker_inputs wrapping library_to_link.
    bazelbuild#10860
  - Toolchain rule is extended with target_settings attribute.
  - --incompatible_restrict_string_escapes=true is now the default.
    Unnecessary backslashes such as "\." in string literals are now
    an error, instead of being silently treated as "\\.".
    To fix the error while preserving behavior, double the backlash.
    However, the error is often a sign that the original code was
    wrong.
  - Propagate instrumented files for transitive sources of
    `android_library` and `android_binary`
  - --local_resources and --incompatible_remove_local_resources have
    been removed. If you've been setting --local resources or
    --incompatible_remove_local_resources=false, you must migrate to
    using --local_ram_resources and --local_cpu_resources instead.
  - Update rules_cc to commit b1c40e1de81913a3c40e5948f78719c28152486d
  - --incompatible_avoid_conflict_dlls=true is now the default.
  - Dynamic execution now uses the new scheduler by default.
  - Dynamic execution now uses the new scheduler by default.
  - The new dynamic scheduler is now the default.

This release contains contributions from many people at Google, as well as Adam Liddell, Akira Baruah, Alexander Grund, Alex Eagle, Andrew Z Allen, Austin Schuh, Benjamin Peterson, Benson Muite, Brentley Jones, Cristian Hancila, Dan Halperin, Daniel Wagner-Hall, Dmitry Ivankov, Dmitry Ivankov, erenon, Eric Cousineau, Greg Estren, Gregor Jasny, Grzegorz Lukasik, Grzegorz Lukasik, hollste, Joe Lencioni, johnjbarton, Jonathan Perry, Jonathon Belotti, Keith Smiley, Kevin Gessner, Matt Davis, Matt Mackay, Menny Even Danan, Neeraj Gupta, Philipp Schrader, Ricardo Delfin, Ryan Beasley, Samuel Giddins, Simon Bjorklen, Simon Stewart, Stiopa Koltsov, Thi Doan, ThomasCJY, Timothy Klim, Tom de Goede, vectoralpha, V Vn Ngha, William A Rowe Jr, Xavier Bonaventura, Yannic Bonenberger, Yannic.
geoffromer added a commit to carbon-language/carbon-lang that referenced this issue Mar 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Performance Issues for Performance teams
Projects
None yet
Development

No branches or pull requests

8 participants