pageserver: tweak period of imitate_layer_accesses

[jcsp]

Problem

When the eviction threshold is an integer multiple of the eviction period, it is unreliable to skip imitating accesses based on whether the last imitation was more recent than the threshold.

This is because as finite time passes
between the time used for the periodic execution, and the 'now' time used for updating last_layer_access_imitation. When this is just a few milliseconds, and everything else is on-time, then a 5 second threshold with a 1 second period will end up entering its 5th iteration slightly less than 5 second since last_layer_access_imitation, and thereby skipping instead of running the imitation. If a few milliseconds then pass before we check the access time of a file that should have been bumped by the imitation pass, then we end up evicting something we shouldn't have evicted.

Summary of changes

We can make this race far less likely by using the threshold minus one interval as the period for re-executing the imitate_layer_accesses: that way we're not vulnerable to racing by just a few millis, and there would have to be a delay of the order period to cause us to wrongly evict a layer.

This is not a complete solution: it would be good to revisit this and use a non-walltime mechanism for pinning these layers into local storage, rather than relying on bumping access times.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[jcsp]

There is a ticket for this test's instability here: #4154 although the analysis on that ticket suggests that the underlying issue was blocking I/Os which this period-tweaking might not help with if a system was heavily loaded enough.

This might also help with #4040, as it looks like the symptom of that was unexpected evictions of things that should have been touched by imitate_layer_accesses.

[github-actions]

1240 tests run: 1189 passed, 0 failed, 51 skipped (full report)

---

Flaky tests (1)

Postgres 14

• test_get_tenant_size_with_multiple_branches: debug

[koivunej]

Inclined to try this regardless of me really seeing how helpful it could be; the only risk is slightly more imitation and I doubt we can measure the increase as it is right now.