pageserver: verify correct handling of VM and FSM pages

[erikgrinaker]

In #9855, we found that VM pages were handled suboptimally and potentially incorrectly.

The VM (visibility map) and FSM (free space map) are stored as forks of the main relation, and their pages are sharded and managed just like regular relation pages. However, updates to the VM and FSM pages may happen differently from regular pages -- they are often updated implicitly on writes to the main relation.

For VM pages specifically, we found in #9855 that ClearVmBits writes were sent to all shards, regardless of whether the shards were responsible for VM pages. These writes would succeed on shards that had VM pages and also on shard 0, but fail and be ignored on other shards (potentially causing significant performance overhead due to relsize cache misses).

In #9895, we fix ClearVmBits such that it's only applied on the shard responsible for the relevant VM pages. However, we need to verify that this is correct, and add appropriate test coverage. We also need to clean up any old partial writes due to misrouting of ClearVmBits (in particular to shard 0). The behavior following #9895 relies on a few assumptions:

• The compute (vacuum) always explicitly writes VM pages via the WAL before any ClearVmBits on those pages.
• The VM shard(s) have a view of the VM relation size that covers all local VM pages.
• We never attempt to access (incomplete) VM pages on shards other than the owning one (including 0).

See also #9855 (comment).

We should also verify that FSM pages are handled similarly and correctly.

[erikgrinaker]

The pg_visibility extension is useful for debugging the visibility map. The following will verify that it's correct (i.e. matches the actual heap pages):

select pg_check_visible(table);
select pg_check_frozen(table);

This was verified by discarding ClearVmBits operations, which was detected by pg_check_visible().

Note that VM pages will often be cached in the compute, so to verify that storage has the correct view the compute must be restarted to clear its caches.

[erikgrinaker]

The following produced 386 faulty VM pages when run with ClearVmBits disabled:

createdb -p 55432 pgbench
./pg_install/v16/bin/pgbench -p 55432 -i -s 100 pgbench
./pg_install/v16/bin/pgbench -p 55432 -c 64 -T 60 -j 16 -P 1 pgbench
cargo neon endpoint stop main
cargo neon endpoint start main
psql -p 55432 pgbench -c "select count(*) from pg_check_visible('pgbench_accounts')"

I'll do five runs across 8 shards with stripe size 16 MB for 1 minute each, before and after #9895, to verify that there is no correctness issue.

[erikgrinaker]

Tests didn't show any VM page corruption.

I've spent the day reading and debugging code, running tests, and talking to Heikki (who wrote the visibility map), and I'm reasonably confident that we don't have a correctness problem. We likely have some stray keys to clean up though, split out #9927 for this.

I'll leave this issue open to consider adding some more assertions, checks, or tests around this.

[erikgrinaker]

Closing this out, the following should address any gaps:

• test_runner: add visibility map test #9940
• pageserver: assert that keys belong to shard #9943

[erikgrinaker]

We found some potentially incorrect FSM key handling over in #10027.