query: add partition labels flag #7722
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MichaHoffmann
force-pushed
the
mhoffmann/add-partition-labels-flag
branch
from
b356e9b
to
83e8df7
Compare
MichaHoffmann
force-pushed
the
mhoffmann/add-partition-labels-flag
branch
from
83e8df7
to
65ff47f
Compare
The distributed engine decides when to push down certain operations by checking if the external labels are still present, i.e. we can push down a binary operation if its vector matching includes all external labels. This is great but if you have multiple external labels that are irrelevant for the partition this is problematic since query authors must be aware of those irrelevant labels and must incorporate them into their queries. This PR attempts to solve that by giving an option to focus on the labels that are relevant for the partition. Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
MichaHoffmann
force-pushed
the
mhoffmann/add-partition-labels-flag
branch
from
65ff47f
to
19e384b
Compare
fpetkovski
reviewed
Sep 12, 2024
Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
MichaHoffmann
force-pushed
the
mhoffmann/add-partition-labels-flag
branch
from
9746716
to
e917201
Compare
fpetkovski
approved these changes
Sep 12, 2024
jnyi
pushed a commit
to jnyi/thanos
that referenced
this pull request
Oct 17, 2024
* query: add partition labels flag The distributed engine decides when to push down certain operations by checking if the external labels are still present, i.e. we can push down a binary operation if its vector matching includes all external labels. This is great but if you have multiple external labels that are irrelevant for the partition this is problematic since query authors must be aware of those irrelevant labels and must incorporate them into their queries. This PR attempts to solve that by giving an option to focus on the labels that are relevant for the partition. Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> * Update cmd/thanos/query.go Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> --------- Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The distributed engine decides when to push down certain operations by checking if the external labels are still present, i.e. we can push down a binary operation if its vector matching includes all external labels. This is great but if you have multiple external labels that are irrelevant for the partition this is problematic since query authors must be aware of those irrelevant labels and must incorporate them into their queries.
This PR attempts to solve that by giving an option to focus on the labels that are relevant for the partition.
Example: if you have
regionanddatacenterexternal labels and everydatacenterhas a distinct label already then you could stripregionfor the decision if we can push something down. For exampleA * on (datacenter) Bwould be able to be pushed down but natively the engine would require it to beA * on (region, datacenter) Bwhich is redundant somewhat.An alternative that doesnt need new code would be to register the unnecessary labels as replica labels but it feels wrong since they dont correspond to any replicas and "region" is a horrible replica label.
Changes
--query.partition-labelflagVerification