Empty depsets should be optimized out of structs the same as they are optimized out of providers #19380
Labels
team-Starlark-Integration
Issues involving Bazel's integration with Starlark, excluding builtin symbols
team-Starlark-Interpreter
Issues involving the Starlark interpreter used by Bazel
type: feature request
untriaged
Comments
iancha1992
added
team-Starlark-Interpreter
Closed
fmeum
added a commit
to fmeum/bazel
that referenced
this issue
Sep 7, 2023
Empty `Depset`s are interned per order, which decreases allocations as well as retained heap when a `Depset` is stored in a `struct` in a provider (providers with schema already unwrap most `Depset`s). Fixes bazelbuild#19380 Closes bazelbuild#19387. PiperOrigin-RevId: 563104923 Change-Id: If66fb4a108ef7569a4f95e7af3a74ac84d1c4636
iancha1992
pushed a commit
that referenced
this issue
Sep 11, 2023
Empty `Depset`s are interned per order, which decreases allocations as well as retained heap when a `Depset` is stored in a `struct` in a provider (providers with schema already unwrap most `Depset`s). Fixes #19380 Closes #19387. PiperOrigin-RevId: 563104923 Change-Id: If66fb4a108ef7569a4f95e7af3a74ac84d1c4636
|
A fix for this issue has been included in Bazel 6.4.0 RC1. Please test out the release candidate and report any issues as soon as possible. Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description of the feature request:
Empty depsets are optimized out of providers, but not structs. This can lead to increased memory usage for some code patterns.
Which category does this issue belong to?
Starlark Integration, Starlark Interpreter
What underlying problem are you trying to solve with this feature?
In rules_xcodeproj we marshal a lot of data collected in aspects down the dependency tree. When doing Starlark memory profiling we noticed that creating new empty
depsetinstances would drastically increase our memory usage. So instead we create frozen instances of empty values and use them when possible: https://github.com/MobileNativeFoundation/rules_xcodeproj/blob/31a9e114dac0e20625fb7d9119fe5a117ecb3608/xcodeproj/internal/memory_efficiency.bzl, which helped us reduce our memory usage.Which operating system are you running Bazel on?
macOS 13.4.1 (c)
What is the output of
bazel info release?release 6.3.2
If
bazel info releasereturnsdevelopment versionor(@non-git), tell us how you built Bazel.No response
What's the output of
git remote get-url origin; git rev-parse master; git rev-parse HEAD?No response
Have you found anything relevant by searching the web?
No response
Any other information, logs, or outputs that you want to share?
https://bazelbuild.slack.com/archives/CA31HN1T3/p1693484609534579 where @fmeum requested I open this issue.
The text was updated successfully, but these errors were encountered: