Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let store-gateways ignore blocks that are too young. #502

Merged
merged 9 commits into from
Nov 18, 2021
4 changes: 3 additions & 1 deletion docs/blocks-storage/querier.md
Original file line number Diff line number Diff line change
Expand Up @@ -733,7 +733,9 @@ blocks_storage:

# Blocks with minimum time within this duration are ignored, and not loaded
# by store-gateway. Useful when used together with
# -querier.query-store-after to prevent loading young blocks.
# -querier.query-store-after to prevent loading young blocks, because there
# is usually many of them (depending on number of ingesters) and they are
pstibrany marked this conversation as resolved.
Show resolved Hide resolved
# not yet compacted.
# CLI flag: -blocks-storage.bucket-store.ignore-blocks-within
[ignore_blocks_within: <duration> | default = 0s]

Expand Down
4 changes: 3 additions & 1 deletion docs/blocks-storage/store-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -786,7 +786,9 @@ blocks_storage:

# Blocks with minimum time within this duration are ignored, and not loaded
# by store-gateway. Useful when used together with
# -querier.query-store-after to prevent loading young blocks.
# -querier.query-store-after to prevent loading young blocks, because there
# is usually many of them (depending on number of ingesters) and they are
pstibrany marked this conversation as resolved.
Show resolved Hide resolved
# not yet compacted.
# CLI flag: -blocks-storage.bucket-store.ignore-blocks-within
[ignore_blocks_within: <duration> | default = 0s]

Expand Down
3 changes: 2 additions & 1 deletion docs/configuration/config-file-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -4987,7 +4987,8 @@ bucket_store:

# Blocks with minimum time within this duration are ignored, and not loaded by
# store-gateway. Useful when used together with -querier.query-store-after to
# prevent loading young blocks.
# prevent loading young blocks, because there is usually many of them
pstibrany marked this conversation as resolved.
Show resolved Hide resolved
# (depending on number of ingesters) and they are not yet compacted.
# CLI flag: -blocks-storage.bucket-store.ignore-blocks-within
[ignore_blocks_within: <duration> | default = 0s]

Expand Down
2 changes: 1 addition & 1 deletion pkg/storage/tsdb/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,7 @@ func (cfg *BucketStoreConfig) RegisterFlags(f *flag.FlagSet) {
f.DurationVar(&cfg.IgnoreDeletionMarksDelay, "blocks-storage.bucket-store.ignore-deletion-marks-delay", time.Hour*6, "Duration after which the blocks marked for deletion will be filtered out while fetching blocks. "+
"The idea of ignore-deletion-marks-delay is to ignore blocks that are marked for deletion with some delay. This ensures store can still serve blocks that are meant to be deleted but do not have a replacement yet. "+
"Default is 6h, half of the default value for -compactor.deletion-delay.")
f.DurationVar(&cfg.IgnoreBlocksWithin, "blocks-storage.bucket-store.ignore-blocks-within", 0, "Blocks with minimum time within this duration are ignored, and not loaded by store-gateway. Useful when used together with -querier.query-store-after to prevent loading young blocks.")
f.DurationVar(&cfg.IgnoreBlocksWithin, "blocks-storage.bucket-store.ignore-blocks-within", 0, "Blocks with minimum time within this duration are ignored, and not loaded by store-gateway. Useful when used together with -querier.query-store-after to prevent loading young blocks, because there is usually many of them (depending on number of ingesters) and they are not yet compacted.")
pstibrany marked this conversation as resolved.
Show resolved Hide resolved
f.IntVar(&cfg.PostingOffsetsInMemSampling, "blocks-storage.bucket-store.posting-offsets-in-mem-sampling", DefaultPostingOffsetInMemorySampling, "Controls what is the ratio of postings offsets that the store will hold in memory.")
f.BoolVar(&cfg.IndexHeaderLazyLoadingEnabled, "blocks-storage.bucket-store.index-header-lazy-loading-enabled", false, "If enabled, store-gateway will lazy load an index-header only once required by a query.")
f.DurationVar(&cfg.IndexHeaderLazyLoadingIdleTimeout, "blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout", 20*time.Minute, "If index-header lazy loading is enabled and this setting is > 0, the store-gateway will offload unused index-headers after 'idle timeout' inactivity.")
Expand Down