Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: suppress noisy "layer became visible" logs #9064

Merged
merged 3 commits into from
Sep 20, 2024

Conversation

jcsp
Copy link
Collaborator

@jcsp jcsp commented Sep 19, 2024

Problem

When layer visibility was added, an info log was included for the situation where actual access to a layer disagrees with the visibility calculation. This situation is safe, but I was interested in seeing when it happens.

The log is pretty high volume, so this PR refines it to fire less often.

Summary of changes

  • For cases where accessing non-visible layers is normal, don't log at all.
  • Extend a unit test to increase confidence that the updates to visibility on access are working as expected
  • During compaction, only call the visibility calculation routine if some image layers were created: previously, frequent calls resulted in the visibility of layers getting reset every time we passed through create_image_layers.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@jcsp jcsp added c/storage/pageserver Component: storage: pageserver a/tech_debt Area: related to tech debt labels Sep 19, 2024
Copy link

github-actions bot commented Sep 19, 2024

4994 tests run: 4830 passed, 0 failed, 164 skipped (full report)


Flaky tests (6)

Postgres 17

Postgres 16

Code coverage* (full report)

  • functions: 31.9% (7424 of 23300 functions)
  • lines: 49.8% (59749 of 119917 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
53437f7 at 2024-09-20T16:17:15.785Z :recycle:

@jcsp jcsp force-pushed the jcsp/visible-log-spam branch from aee8943 to 53437f7 Compare September 19, 2024 16:32
@jcsp jcsp marked this pull request as ready for review September 20, 2024 11:36
@jcsp jcsp requested a review from a team as a code owner September 20, 2024 11:36
@jcsp jcsp requested a review from yliang412 September 20, 2024 11:36
@jcsp jcsp enabled auto-merge (squash) September 20, 2024 15:08
@jcsp jcsp merged commit 6014f15 into main Sep 20, 2024
79 checks passed
@jcsp jcsp deleted the jcsp/visible-log-spam branch September 20, 2024 16:07
davidgomes pushed a commit that referenced this pull request Sep 21, 2024
## Problem

When layer visibility was added, an info log was included for the
situation where actual access to a layer disagrees with the visibility
calculation. This situation is safe, but I was interested in seeing when
it happens.

The log is pretty high volume, so this PR refines it to fire less often.

## Summary of changes

- For cases where accessing non-visible layers is normal, don't log at
all.
- Extend a unit test to increase confidence that the updates to
visibility on access are working as expected
- During compaction, only call the visibility calculation routine if
some image layers were created: previously, frequent calls resulted in
the visibility of layers getting reset every time we passed through
create_image_layers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/tech_debt Area: related to tech debt c/storage/pageserver Component: storage: pageserver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants