Metro Cache with S3

[afoxman]

AirBNB: Rae Liu, Noah Sugarmann
Recording: Link

Metro Cache docs: https://facebook.github.io/metro/docs/caching

The cache tracks Babel-transformed files and the Metro graph.

AirBNB uses three levels of caching:

• In-memory cache -- custom implementation using a simple in-memory dictionary. Metro doesn't offer this.
• FileStore -- Metro impl exists. You configure it with a path that it uses as a dumping ground.
• NetworkStore -- they were using S3, but switched to an internal system/proxy to speed things up (my concern is securing the data on S3 and managing access)

Network store is based on Metro's HTTP store

Compression

gzip files on set and gunzip on get

Access

Devs can READ, CI can READ and WRITE

Official builds create cache baselines
Dev machines have local file and memory caches to store deltas.

Warning: writes generate more traffic than reads and scoping to CI seems to be the right balance

Pruning

didn't discuss this

File store: need to see if/how Metro manages this. resetCache option in CLI?
Network store: will need a date/size-based policy, likely implemented as a scheduled CI job

Keys

metro creates these and you can supply additional material to the hash generator

AirBNB had to include additional details like such as their babel transforms and all of the dependencies that go into them. In some cases, they included source code.

They had some issues with relative paths vs. absolute paths. Especially in their custom module resolution logic.

Careful with this on Windows vs Mac vs Linux. Git transforms newlines, for example.

Warning: Key generation was the trickiest area to get right. Will need to pay careful attention.

Metrics

AirBNB has a wrapper for all get/set calls to track hit rates. Results are published to a shared DB and metrics are tallied and examined automatically. Alerts are triggered on low hit rates.