Skip to content

Commit

Permalink
Added building blocks for dynamic wear-leveling
Browse files Browse the repository at this point in the history
Initially, littlefs relied entirely on bad-block detection for
wear-leveling. Conceptually, at the end of a devices lifespan, all
blocks would be worn evenly, even if they weren't worn out at the same
time. However, this doesn't work for all devices, rather than causing
corruption during writes, wear reduces a devices "sticking power",
causing bits to flip over time. This means for many devices, true
wear-leveling (dynamic or static) is required.

Fortunately, way back at the beginning, littlefs was designed to do full
dynamic wear-leveling, only dropping it when making the retrospectively
short-sighted realization that bad-block detection is theoretically
sufficient. We can enable dynamic wear-leveling with only a few tweaks
to littlefs. These can be implemented without breaking backwards
compatibility.

1. Evict metadata-pairs after a certain number of writes. Eviction in
   this case is identical to a relocation to recover from a bad block.
   We move our data and stick the old block back into our pool of
   blocks.

   For knowing when to evict, we already have a revision count for each
   metadata-pair which gives us enough information. We add the
   configuration option block_cycles and evict when our revision count
   is a multiple of this value.

2. Now all blocks participate in COW behaviour. However we don't store
   the state of our allocator, so every boot cycle we reuse the first
   blocks on storage. This is very bad on a microcontroller, where we
   may reboot often. We need a way to spread our usage across the disk.

   To pull this off, we can simply randomize which block we start our
   allocator at. But we need a random number generator that is different
   on each boot. Fortunately we have a great source of entropy, our
   filesystem. So we seed our block allocator with a simple hash of the
   CRCs on our metadata-pairs. This can be done for free since we
   already need to scan the metadata-pairs during mount.

What we end up with is a uniform distribution of wear on storage. The
wear is not perfect, if a block is used for metadata it gets more wear,
and the randomization may not be exact. But we can never actually get
perfect wear-leveling, since we're already resigned to dynamic
wear-leveling at the file level.

With the addition of metadata logging, we end up with a really
interesting two-stage wear-leveling algorithm. At the low-level,
metadata is statically wear-leveled. At the high-level, blocks are
dynamically wear-leveled.

---

This specific commit implements the first step, eviction of metadata
pairs. Entertwining this into the already complicated compact logic was
a bit annoying, however we can combine the logic for superblock
expansion with the logic for metadata-pair eviction.
  • Loading branch information
geky committed Oct 18, 2018
1 parent 20b669a commit e4a0d58
Show file tree
Hide file tree
Showing 4 changed files with 317 additions and 267 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ script:

# run tests with a few different configurations
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=1 -DLFS_CACHE_SIZE=4"
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=512 -DLFS_CACHE_SIZE=512"
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=512 -DLFS_CACHE_SIZE=512 -DLFS_BLOCK_CYCLES=16"
- make test QUIET=1 CFLAGS+="-DLFS_BLOCK_COUNT=1023 -DLFS_LOOKAHEAD=2048"

- make clean test QUIET=1 CFLAGS+="-DLFS_INLINE_MAX=0"
Expand Down
Loading

0 comments on commit e4a0d58

Please sign in to comment.