pageserver: avoid incrementing access time when reading layers for compaction

[jcsp]

Problem

Currently, image generation reads delta layers before writing out subsequent image layers, which updates the access time of the delta layers and effectively puts them at the back of the queue for eviction. This is the opposite of what we want, because after a delta layer is covered by a later image layer, it's likely that subsequent reads of latest data will hit the image rather than the delta layer, so the delta layer should be quite a good candidate for eviction.

Summary of changes

RequestContext gets a new ATimeBehavior field, and a RequestContextBuilder helper so that we can optionally add the new field without growing RequestContext::new every time we add something like this.

Request context is passed into the record_access function, and the access time is not updated if ATimeBehavior::Skip is set.

The compaction background task constructs its request context with this skip policy.

Closes: #4969

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

1588 tests run: 1510 passed, 0 failed, 78 skipped (full report)

---

_{The comment gets automatically updated with the latest test results
d2bf8f5 at 2023-08-14T08:19:19.339Z :recycle:}

[koivunej]

I think this could just be an if in DeltaLayer::load in

neon/pageserver/src/tenant/storage_layer/delta_layer.rs

Lines 455 to 456 in d39fd66

	self.access_stats
	.record_access(access_kind, ctx.task_kind());

Like:

if access_kind != AccessKind::CreateImageLayers {
    self.access_stats.record_access(access_kind, ctx.task_kind());
}

After this PR, we could already try 1TB import in staging to see if any needle is moved.

[jcsp]

I don't think a LayerAccessKind variant works for this. The reads in create_image_layers go via the general purpose get(), and ultimately get_value_reconstruct_data -- it doesn't take LayerAccessKind arguments (but it does carry the context object).

I'd also not want to make this behavior specific to delta layer, because image layer generation will also depend on reading earlier image layers, where we'd also like to avoid updating their access times.

[koivunej]

My bad for not giving this a more thorough look last friday. Approved with getting rid of "atime" mention just not to confuse with the fs level information which we explicitly ended up not using.

[jcsp]

Added a commit that tweaks the names to avoid potential confusion around filesystem atime vs out internal access stats.