Skip to content

Commit

Permalink
chore(blooms)!: Remove bloom compactor component (#13969)
Browse files Browse the repository at this point in the history
**What this PR does / why we need it**:

This commit removes the code related to the bloom compactor which is superseded by the bloom planner and builders.

A handful of CLI arguments of per-tenant settings changed their prefix from `-bloom-compactor.*` to `-bloom-build.*`. Other settings for the compactor component itself, also prefixed with `-bloom-compactor.*` were removed. See upgrade docs for further information.

**Special notes for your reviewer**:

:heavy_check_mark: Part of #13957

📔 Documentation update #13965

⚠️ Integrating the bloom planner and builder into the `backend` target is done in a follow-up PR.

---

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
  • Loading branch information
chaudum authored Aug 29, 2024
1 parent 6731ea9 commit b75eacc
Show file tree
Hide file tree
Showing 39 changed files with 175 additions and 6,786 deletions.
4 changes: 2 additions & 2 deletions docs/sources/operations/query-acceleration-blooms.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ and querying the bloom filters that only pays off at large scale deployments.
To start building and using blooms you need to:
- Deploy the [Bloom Planner and Builder](#bloom-planner-and-builder) components and enable the component in the [Bloom Build config][bloom-build-cfg].
- Deploy the [Bloom Gateway](#bloom-gateway) component (as a [microservice][microservices] or via the [SSD][ssd] Backend target) and enable the component in the [Bloom Gateway config][bloom-gateway-cfg].
- Enable blooms filtering and compaction for each tenant individually, or for all of them by default.
- Enable blooms building and filtering for each tenant individually, or for all of them by default.

```yaml
# Configuration block for the bloom creation.
Expand Down Expand Up @@ -119,7 +119,7 @@ Builders process tasks sequentially by pulling them from the queue. The amount o
all pending tasks before the next planning iteration depends on the value of `-bloom-build.planner.bloom_split_series_keyspace_by`,
the amount of tenants, and the log volume of the streams.

The maximum block size is configured per tenant via `-bloom-compactor.max-block-size`.
The maximum block size is configured per tenant via `-bloom-build.max-block-size`.
The actual block size might exceed this limit given that we append streams blooms to the block until the
block is larger than the configured maximum size. Blocks are created in memory and as soon as they are written to the
object store they are freed. Chunks and TSDB files are downloaded from the object store to the file system.
Expand Down
30 changes: 30 additions & 0 deletions docs/sources/setup/upgrade/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,36 @@ parameter contains a log selector query instead of returning inconsistent result

Loki changes the default value of `-ruler.alertmanager-use-v2` from `false` to `true`. Alertmanager APIv1 was deprecated in Alertmanager 0.16.0 and is removed as of 0.27.0.

### Experimental Bloom Filters

{{% admonition type="note" %}}
Experimental features are subject to rapid change and/or removal, which can introduce breaking changes even between minor version.
They also don't follow the deprecation lifecycle of regular features.
{{% /admonition %}}

The bloom compactor component, which builds bloom filter blocks for query acceleration, has been removed in favor of two new components: bloom planner and bloom builder.
Please consult the [Query Acceleration with Blooms](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/query-acceleration-blooms/) docs for more information.

CLI arguments (and their YAML counterparts) of per-tenant settings that have been removed as part of this change:

* `-bloom-compactor.enable-compaction`
* `-bloom-compactor.shard-size`
* `-bloom-compactor.shard-size`
* `-bloom-compactor.shard-size`

CLI arguments of per-tenant settings that have been moved to a different prefix as part of this change:

* `-bloom-compactor.max-page-size` changed to `-bloom-builder.max-page-size`
* `-bloom-compactor.max-block-size` changed to `-bloom-builder.max-block-size`
* `-bloom-compactor.ngram-length` changed to `-bloom-builder.ngram-length`
* `-bloom-compactor.ngram-skip` changed to `-bloom-builder.ngram-skip`
* `-bloom-compactor.false-positive-rate` changed to `-bloom-builder.false-positive-rate`
* `-bloom-compactor.block-encoding` changed to `-bloom-builder.block-encoding`

Their YAML counterparts in the `limits_config` block are kept identical.

All other CLI arguments (and their YAML counterparts) prefixed with `-bloom-compactor.` have been removed.

## 3.0.0

{{% admonition type="note" %}}
Expand Down
Loading

0 comments on commit b75eacc

Please sign in to comment.