Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge release-0.19 to main. #4008

Merged
merged 17 commits into from
Apr 9, 2021
Merged

Merge release-0.19 to main. #4008

merged 17 commits into from
Apr 9, 2021

Conversation

bwplotka
Copy link
Member

@bwplotka bwplotka commented Apr 1, 2021

NOTE: Use merge commit not squash to merge this PR.

bwplotka and others added 15 commits March 2, 2021 09:16
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Missed from: #3860

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: yeya24 <yb532204897@gmail.com>
* Fix bug introduced in PR #3740

Signed-off-by: Hitanshu Mehta <hitanshu99amehta@gmail.com>

* Minor change in test

Signed-off-by: Hitanshu Mehta <hitanshu99amehta@gmail.com>

Co-authored-by: Hitanshu Mehta <44025541+hitanshu-mehta@users.noreply.github.com>
…rs (#3886) (#3890)

* rewrite delete should delete a series only if it matches all matchers in a deletion request

Signed-off-by: yeya24 <yb532204897@gmail.com>

* add test case

Signed-off-by: yeya24 <yb532204897@gmail.com>

Co-authored-by: Ben Ye <yb532204897@gmail.com>
Currently, the thanos Receiver infers whether it should use TLS for
the gRPC clients that forwards time series to other receivers based on
whether the remote-write HTTP server uses TLS. This is not correct, as
the HTTP server may use TLS without the gRPC server using TLS. This
commit fixes the inference.

Longer-term, this will be taken care of by the receive/router split.

Signed-off-by: Lucas Servén Marín <lserven@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fix partial and empty matchers match

Signed-off-by: yeya24 <yb532204897@gmail.com>

* add testcase for non-equal matchers

Signed-off-by: yeya24 <yb532204897@gmail.com>
…usage introduced in 0.17 (#3943)

* Added receive benchmark, baseline.

```
goos: linux
goarch: amd64
pkg: github.com/thanos-io/thanos/pkg/receive
BenchmarkHandlerReceiveHTTP
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them.
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them./OK
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them./OK-12      	   22260	   1550152 ns/op	 1380340 B/op	    6093 allocs/op
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them./conflict_errors
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them./conflict_errors-12         	    6619	   6430408 ns/op	 4522487 B/op	   26118 allocs/op
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them.
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them./OK
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them./OK-12                     	    2695	  17208794 ns/op	15072963 B/op	   60441 allocs/op
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them./conflict_errors
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them./conflict_errors-12        	     474	  72533286 ns/op	46396932 B/op	  260141 allocs/op
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/OK
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/OK-12                	     270	 137050518 ns/op	226595379 B/op	     132 allocs/op
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/conflict_errors
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/conflict_errors-12   	      21	1616025443 ns/op	698724321 B/op	     408 allocs/op
PASS

Process finished with exit code 0
```


Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Copy labels.

```
GOROOT=/home/bwplotka/.gvm/gos/go1.15 #gosetup
GOPATH=/home/bwplotka/Repos/thanosgopath #gosetup
/home/bwplotka/.gvm/gos/go1.15/bin/go test -c -o /tmp/___BenchmarkHandlerReceiveHTTP_in_github_com_thanos_io_thanos_pkg_receive github.com/thanos-io/thanos/pkg/receive #gosetup
/tmp/___BenchmarkHandlerReceiveHTTP_in_github_com_thanos_io_thanos_pkg_receive -test.v -test.bench ^\QBenchmarkHandlerReceiveHTTP\E$ -test.run ^$ -test.benchmem -test.benchtime=30s
goos: linux
goarch: amd64
pkg: github.com/thanos-io/thanos/pkg/receive
BenchmarkHandlerReceiveHTTP
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/OK
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/OK-12      	   25887	   1537262 ns/op	 1380023 B/op	    6092 allocs/op
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/conflict_errors
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/conflict_errors-12         	    4237	   7547968 ns/op	 4522583 B/op	   26118 allocs/op
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/OK
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/OK-12                     	    2205	  16513380 ns/op	15071092 B/op	   60420 allocs/op
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/conflict_errors
BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/conflict_errors-12        	     525	  67278233 ns/op	46396645 B/op	  260141 allocs/op
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/OK
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/OK-12               	     285	 148049189 ns/op	226596168 B/op	     132 allocs/op
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/conflict_errors
BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/conflict_errors-12  	      20	1731361499 ns/op	698722550 B/op	     401 allocs/op
PASS

Process finished with exit code 0

```

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addded bench.,

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fix.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Improved API.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Changelog.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Lucas comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* compact: clean up directories properly

I couldn't stop thinking about this code for some reason and I have
figured that I had missed one case in #3031. We need to also clean up
the directories in compaction groups. A compaction could fail leaving
some new directory with a random ULID on the disk. Before attempting to
do another compaction loop, we need to remove it as well because the
compaction process always produces a new, unique directory.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* Remove all non expected dirs.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comment.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
…eed. (#3968)

Fixes: #3917

Long term fix: #3967

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
…3960) (#3999)

Currently, if an alerting rule having the same name with different
severity labels is being returned from different replicas then they
are being treated as separate alerts.

Given the following alerts a1,a2 with severities s1,s2 returned from
replicas r1,2:

a1[s1,r1]
a1[s2,r1]
a1[s1,r2]
a1[s2,r2]

Then, currently, the algorithm deduplicates to:

a1[s1]
a1[s2]
a1[s1]
a1[s2]

Instead of the intendet result:

a1[s1]
a1[s2]

This fixes it by removing replica labels before sorting labels for
deduplication.

Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com>
# Conflicts:
#	CHANGELOG.md

Co-authored-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
@bwplotka bwplotka requested review from onprem, yeya24 and squat April 1, 2021 14:10
onprem
onprem previously approved these changes Apr 1, 2021
Copy link
Member

@onprem onprem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@GiedriusS
Copy link
Member

@bwplotka any update on this? 🤗

# Conflicts:
#	CHANGELOG.md
#	VERSION
#	cmd/thanos/receive.go
#	pkg/receive/handler_test.go
@bwplotka
Copy link
Member Author

bwplotka commented Apr 9, 2021

Sorry for lag. Can't work with merge requests 🤦🏽

# Conflicts:
#	pkg/metadata/prometheus_test.go
#	test/e2e/metadata_api_test.go
@bwplotka bwplotka merged commit 7dc2f3d into main Apr 9, 2021
@bwplotka bwplotka deleted the merge-v0.19 branch April 9, 2021 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants