Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce cleaner visit marker #6113

Merged
merged 18 commits into from
Jul 31, 2024

Conversation

alexqyle
Copy link
Contributor

@alexqyle alexqyle commented Jul 24, 2024

What this PR does:

Introduce cleaner visit marker to make sure cleaning cycle for one tenant can only be run on one cleaner at a time. With visit marker, there is an extra check to make sure one tenant will not be run on two cleaner at same time. Also, this change could potentially remove sharding for cleaner. So that any cleaner could take any tenant for cleaning cycle.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

…nant can only be run on one cleaner at a time

Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
CHANGELOG.md Outdated Show resolved Hide resolved
Signed-off-by: Alex Le <leqiyue@amazon.com>
alexqyle added 4 commits July 24, 2024 12:15
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
Copy link
Contributor

@danielblando danielblando left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

Copy link
Contributor

@harry671003 harry671003 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of my comments are minor/cosmetic to improve the readability. Feel free to push back.

pkg/compactor/blocks_cleaner.go Outdated Show resolved Hide resolved
pkg/compactor/visit_marker.go Outdated Show resolved Hide resolved
pkg/compactor/compactor.go Outdated Show resolved Hide resolved
pkg/compactor/visit_marker.go Outdated Show resolved Hide resolved
pkg/compactor/visit_marker.go Outdated Show resolved Hide resolved
pkg/compactor/blocks_cleaner.go Outdated Show resolved Hide resolved
pkg/compactor/blocks_cleaner.go Show resolved Hide resolved
pkg/compactor/blocks_cleaner.go Outdated Show resolved Hide resolved
pkg/compactor/blocks_cleaner.go Outdated Show resolved Hide resolved
pkg/compactor/visit_marker.go Outdated Show resolved Hide resolved
alexqyle added 2 commits July 29, 2024 14:38
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
alexqyle added 2 commits July 30, 2024 16:12
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
Copy link
Contributor

@harry671003 harry671003 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes.

alexqyle added 5 commits July 30, 2024 17:27
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @alexqyle. I added some comments but most of them are nits, non blocking at all

pkg/compactor/visit_marker.go Show resolved Hide resolved
pkg/compactor/visit_marker.go Show resolved Hide resolved
pkg/compactor/visit_marker.go Outdated Show resolved Hide resolved
pkg/compactor/visit_marker.go Outdated Show resolved Hide resolved
pkg/compactor/compactor.go Outdated Show resolved Hide resolved
pkg/compactor/blocks_cleaner.go Outdated Show resolved Hide resolved
pkg/compactor/visit_marker_test.go Outdated Show resolved Hide resolved
pkg/compactor/blocks_cleaner.go Outdated Show resolved Hide resolved
alexqyle added 2 commits July 31, 2024 09:41
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <leqiyue@amazon.com>
@yeya24 yeya24 merged commit 4539330 into cortexproject:master Jul 31, 2024
16 checks passed
@alexqyle alexqyle deleted the cleaner-visit-marker branch July 31, 2024 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants