Hub Federation (client-side)

[lyoshenka]

There are two related issues: hubs connecting to other hubs and advertising themselves, and clients discovering other hubs to connect to.

Before starting any of this, look at how current Electrum server federation works. Look at server.peer.subscribe. It may already do most of this, and may actually work better than described below. Most likely we'll have to make modifications to what's there, for example for jurisdictions.

user stories

As a hub operator, in order to avoid doing work to get ppl to use my server, I want to have my server be automatically discoverable by others.

As a LBRY user, in order to trust the network and have a good experience, I want a robust system of hubs that doesn't depend on LBRY Inc to run them all.

part 1 - clients learning from hubs

on the hub

• hardcode a list of other known hub peers. this is temporary. the list will become dynamic in part 2

on the client

• when client connects to hub, it subscribes to be notified about the list and changes to it
• merge these hubs into the client's list of known hubs
• store known hubs on disk in a known_hubs file
• load stored hubs from disk on client startup
• if custom hubs are set in config, only use those. if not, use all known hubs.
• if old "wallet_servers" config value is set to exactly the default hubs, delete it from config
• add "jurisdiction" config. only use hubs that match the client jurisdiction.
• test like this: have one preset hub, connect to it, discover one more hub, disconnect from first hub. this should cause client to try to connect to the newly-discovered hub
• test that clients with this feature sitll work with hubs without it
• fallback to TCP for jurisdiction if UDP doesnt work

jack had ideas about how to do the requesting of new hub as part of the udp connection process. i think that's an extra thing that we can implement later.

part 1 can be PRed on it's own

part 2 - hubs learning from other hubs

moved to #3338

later

• add timeouts to remove unresponsive peers
• add a way to check what block peers are on, and drop peers that are way behind

questions

• how to onboard users to setting a jurisdiction?
  • guess one based on ip/locale
  • make user choose?
  • use None? not great to have this edge case, but need to distinguish between something the user selected vs something we guessed for them
• how to handle jurisdictions

hub config algo v1

• if you have anything in your config, select the lowest-latency hub from among the ones in the config
• if config is empty, do discovery (starting from hardcoded hubs)

[lyoshenka]

victor says

this is a matter of flipping a switch on server side and asking for peers on client side, I think its groomed but is blocked by properly verifying proofs because the current real issue is that an "evil" server can send fake data without being caught as of now

proofs pr: #2183

[lyoshenka]

closed by #3300, with the remainder in #3338