-
Notifications
You must be signed in to change notification settings - Fork 179
Don't write empty blocks #374
Don't write empty blocks #374
Conversation
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
d2d6cd1
to
beb0d73
Compare
LGTM |
actually I just thought it will be good to add a test for this with some comments why we don't want empty blocks. |
Yup, a test for this would be good. LGTM otherwise. |
And the test found out a bug. The compaction of head was going on in an infinite loop, but not sure why. One workaround that I tested (which worked properly) was that, send the dir of empty block in Does this sound fine? |
can you add the test so I can check what is happening. |
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
@krasi-georgiev I added the test case. |
https://github.com/prometheus/tsdb/blob/2945db18cafdef58d9c511d4c9f223becb69ac5a/db.go#L374-L376 it never exits the loop because the head range is not reset.
I think 1 makes more sense. |
We never add "empty" sample :) We can delete samples though. It's kinda hard to set the mintime during |
@codesome still stuck with the test? |
@krasi-georgiev Stuck with exams 😛. I will be back on it tomorrow. |
Thanks! |
EDIT: You can skip this comment. Not relevant anymore. @gouthamve @krasi-georgiev This allows you to write empty block during compact, but removes it immediately after. More like, empty blocks might appear during |
I would say lets spend more time in trying to fix the test. ping me if you run out if ideas and I can also have a look. |
@krasi-georgiev Actually the test is not broken, the code is broken. I actually found a fix for it (tested, working), I will push it soon. |
When number of blocks becomes 0, head is not truncated in reload(). This fixes it. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
With previous design if a block is not created from parent blocks, the parent block gets undeleted. This adds `Deletable` field in `BlockMetaCompaction` which helps find such parent blocks. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
@krasi-georgiev This is good for anther review. |
lets also update the comment for the
|
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
98b38fd
to
7623f3e
Compare
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Pull/374 review
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Was not able to address this yet: #374 (comment), rest all reviews have been addressed. |
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Pull/374 review
LGTM |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that this could break compactions in subtle ways if the wrong ranges are compacted together. Should return only one block for rewriting tombstones.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
all done |
* Dont write empty blocks when a compaction results in a block with no samples. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
https://github.com/prometheus/prometheus/releases Some of these changes seem to be interesting enough to update [ENHANCEMENT] Query performance improvements. prometheus-junkyard/tsdb#531 [BUGFIX] Scrape: catch errors when creating HTTP clients #5182. Adds new metrics: prometheus_target_scrape_pools_* deprecating the flag storage.tsdb.retention -> use storage.tsdb.retention.time [FEATURE] Add subqueries to PromQL. [ENHANCEMENT] Kubernetes SD: Add service external IP and external name to the discovery metadata. #4940 [ENHANCEMENT] Add metric for number of rule groups loaded. #5090 BUGFIX] Make sure the retention period does not overflow. #5112 [BUGFIX] Make sure the blocks do not get very large. #5112 [BUGFIX] Do not generate blocks with no samples. prometheus-junkyard/tsdb#374 [BUGFIX] Reintroduce metric for WAL corruptions. prometheus-junkyard/tsdb#473 Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
Fixes #309
Supersedes #245