Shared local SSD cache for Velox

[zhouyuan]

Velox local caching feature is very helpful when reading from slow storage(S3 or remote HDFS w/ HDDs), it does not support sharing between executors on one compute node. For Spark applications one typical setup is to have multiple executors running on one compute node, e.g. on a 96-core server people may allocate 24 executors with 4 core per executor. Currently if local SSD cache is enable in this case, the same content maybe access by different executors thus the data will be cached multiple times on that compute node. The cache is not efficiently used. A shared local SSD cache can fix this issue. Here's a rough design for this:

• An SSD Cache Daemon: On each compute node the daemon will control the shared cache state, the daemon will listen on a local domain socket
• Executor instances will do IPC(unix domain socket based) with the daemon do read/write lock on a shared cache block
• Cache blocks from tables(s) are cached in a shared area on compute node(s)
• Reads are served from the shared cache, writes are not supported yet(?)

CC: @oerling @mbasmanova

thanks, -yuan

[Yohahaha]

It looks like Alluxio ?

[zuochunwei]

GOOD