Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add safekeeper scrubber test #8785

Merged
merged 6 commits into from
Aug 30, 2024
Merged

Conversation

arpad-m
Copy link
Member

@arpad-m arpad-m commented Aug 21, 2024

The test is very rudimentary, it only checks that before and after tenant deletion, we can run scan_metadata for the safekeeper node kind. Also, we don't actually expect any uploaded data, for that we don't have enough WAL (needs to create at least one S3-uploaded file, the scrubber doesn't recognize partial files yet).

The scan_metadata scrubber subcommand is extended to support either specifying a database connection string, which was previously the only way, and required a database to be present, or specifying the timeline information manually via json. This is ideal for testing scenarios because in those, the number of timelines is usually limited.

@arpad-m arpad-m requested a review from arssher August 21, 2024 13:25
@arpad-m arpad-m force-pushed the arpad/storage_scrubber_safekeepers_test branch from 5264756 to ea2abf6 Compare August 21, 2024 13:52
Copy link

github-actions bot commented Aug 21, 2024

3827 tests run: 3721 passed, 0 failed, 106 skipped (full report)


Flaky tests (4)

Postgres 16

Postgres 15

Code coverage* (full report)

  • functions: 32.5% (7416 of 22827 functions)
  • lines: 50.6% (60049 of 118663 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
666295e at 2024-08-30T23:08:09.611Z :recycle:

Copy link
Member

@skyzh skyzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @yliang412 might want to add this to the cron job if necessary?

storage_scrubber/src/main.rs Show resolved Hide resolved
storage_scrubber/src/main.rs Outdated Show resolved Hide resolved
@yliang412
Copy link
Contributor

LGTM, @yliang412 might want to add this to the cron job if necessary?

I think we can do it if the checks are stable + do not emit false positive. As part of #8128, I only checked into pageserver's metadata scan code.

@arpad-m arpad-m merged commit 3ec785f into main Aug 30, 2024
68 checks passed
@arpad-m arpad-m deleted the arpad/storage_scrubber_safekeepers_test branch August 30, 2024 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants