How does Kademlia caching _work_?

[joshuef]

So, I've looked to enable KademliaCaching in the kad config. As I understand it anythign Quorum::One should automatically be cached at the closest X nodes to the data...

Now, I'm not really sure how to verify this is happening, or where it happens. I have poked about the code, but didn't turn up anything.

So I'm wondering:

• Is caching automatic?
• Are records cached separately to the record store? (I'd assumed so, but perhaps not?)
• Can the cache size be controlled? And if so, where?
• Can we identify/filter/act on caches records at nodes?

Thanks for any pointers here! 🙇

[mxinden]

Is caching automatic?

It has been in the past with Quorum::One, but is no longer. We have removed the Quorum mechanism from get_record. See #2712. The user needs to manually call Kademlia::put_record_to on a GetRecordOk::FinishedWithNoAdditionalRecord.

Note that the current docs are outdated, as they still reference the Quorum on the get_record code path. Would you mind creating a pull request updating the docs @joshuef?

rust-libp2p/protocols/kad/src/behaviour.rs

Lines 2778 to 2789 in 38ea7ad

	FinishedWithNoAdditionalRecord {
	/// If caching is enabled, these are the peers closest
	/// _to the record key_ (not the local node) that were queried but
	/// did not return the record, sorted by distance to the record key
	/// from closest to farthest. How many of these are tracked is configured
	/// by [`KademliaConfig::set_caching`]. If the lookup used a quorum of
	/// 1, these peers will be sent the record as a means of caching.
	/// If the lookup used a quorum > 1, you may wish to use these
	/// candidates with [`Kademlia::put_record_to`] after selecting
	/// one of the returned records.
	cache_candidates: BTreeMap<kbucket::Distance, PeerId>,
	},

Are records cached separately to the record store? (I'd assumed so, but perhaps not?)

I am not sure I understand the question. Maybe the following helps. The local node looks up a key. It discovers multiple peers that should but don't actually have the key. When the user now calls put_record_to with the key, value and the peers, the local node will attempt to put the record to the peer just as it would with the normal Kademlia::put_record. Just that this time it knows upfront which peers to target. From the peers perspective, it can not differentiate the write-back caching from a normal put_record.

Can the cache size be controlled? And if so, where?

In case you are using the MemoryStore you can use the MemoryStoreConfig::max_records option.

Can we identify/filter/act on caches records at nodes?

No. Remote notes are ignorant of the fact that an incoming put record is actually from a cache write-back.

I am sorry that this is not more intuitive. Hope this helps @joshuef.

[joshuef]

Here's the PR that I think clarifies the documentation: #4564

[mxinden]

As things stablilise I'm hopeful we might be able to bring some of this upstream to you all. At the moment we're still testing and tweaking right now!

Maybe something to present at one of our open maintainer calls or the libp2p community call @joshuef?

[joshuef]

Ah sorry, missed that @mxinden ! Yeh I could talk show you how our replication flow is working at the moment and how/where that's sticking for us or perhaps counter to how libp2p might be intended.

[mxinden]

Great. Let me know if this works for you. libp2p/specs#588

[joshuef]

Yeh that should be fine 👍