Disk usage optimized file placing

[willscott]

Currently, there is not movement of underlying pebble 'slabs' / store the index instances, to separate actively filling activity from "full" shards that are only used for reads.

the latter, if they're stored in a separate mount / folder can be placed on slower spinning disks and make the overall cost of a bare-metal deployment of STI much cheaper to operate

[gammazero]

The full slabs are not completely read-only since data can still be deleted from them. How do we know when they are considered full? We may need to disable deletes when moving data. Maybe store the deletes and play them back later.

Which is the bigger cost savings, moving the data to a lower-grade media (spinning disk) or moving the disk to a lower-grade server?