p2p: smart discovery (dead/solo remedy subnets)

[iurii-ssv]

Replaces #1917 additionally improving a bunch of p2p-handling code, with the main contributing factor being "selective discovery"

Before merging:

• remove/resolve TODOs used for testing
• (when testing) make sure this PR doesn't introduce memory-leaks or super-high CPU consumption type of issues
• do we want to set MinConnectivitySubnets to higher-than-0 value (say 3) ? There isn't really a good way to tell how it affects network connectivity (but it's almost certainly gonna improve it, not worsen it)

[iurii-ssv]

Another thing I've realized is happening - because we only trim once we've reached MaxPeers limit, at some point we might stop trimming completely - for example, we've got to max amount of the incoming connections we allow via 0.5 ratio (say, 30) + we've discovered some outgoing peers but not enough to reach MaxPeers limit,

in other words, we've discovered everything we could (and we keep discovering, just at a very slow rate - due to quite an aggressive filter there isn't many connections we are interested in) but that also means we stopped re-cycling(trimming) our incoming connections periodically because we don't trim incoming/outgoing connections separately

I've added a commit to address that - d230c4a - it's pretty "hacky/ad-hoc" way do the job - ideally (to simplify this) I think we probably want to have separate limits per incoming/outgoing connections and somewhat separate trimming logic, WDYT ?

[codecov]

Codecov Report

Attention: Patch coverage is 25.20000% with 561 lines in your changes missing coverage. Please review.

Project coverage is 47.3%. Comparing base (ff4fa98) to head (82d4805).
Report is 23 commits behind head on stage.

Files with missing lines	Patch %	Lines
network/p2p/p2p.go	6.7%	339 Missing and 4 partials ⚠️
network/commons/subnets.go	55.5%	37 Missing and 7 partials ⚠️
network/p2p/p2p_pubsub.go	2.3%	41 Missing ⚠️
network/discovery/dv5_filters.go	41.9%	14 Missing and 4 partials ⚠️
network/peers/connections/conn_gater.go	0.0%	18 Missing ⚠️
network/p2p/p2p_setup.go	61.3%	14 Missing and 3 partials ⚠️
network/p2p/observability.go	6.2%	15 Missing ⚠️
network/peers/conn_manager.go	0.0%	14 Missing ⚠️
network/peers/connections/conn_handler.go	0.0%	12 Missing ⚠️
network/discovery/dv5_service.go	66.6%	8 Missing and 3 partials ⚠️
... and 11 more

Additional details and impacted files

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[y0sher]

previous location looks better

[iurii-ssv]

I think we need a make format command that would take care of all the imports, it's a bit of time-waste to look out for this manually

Can we do a separate task/PR for this to fix it project-wide ?

cc @moshe-blox @nkryuchkov @oleg-ssvlabs @MatusKysel

I've had success using https://github.com/daixiang0/gci in the past - it's fast and we can make separate sections for golang deps, 3rd-party deps, our own package-deps for example

[nkryuchkov]

@iurii-ssv I agree, we often change imports according to different rules and it causes merge conflicts

[y0sher]

couldn't we just use this pool in the filters to minimize discovering same peer over and over?

[iurii-ssv]

Indeed we can, done - 82d4805

[y0sher]

why not choose from the whole pool?

[iurii-ssv]

The size of peersByPriority (which is slightly trimmed version of the whole current pool of discovered peers peers.DiscoveredPeersPool) can be large - hundreds or even thousands maybe,

from what I understand there isn't a much better solution to this problem than brute-force (which is what I'm doing here) because any combination might be best - we don't know until we compare it against the best we've got so far

so from my testing I figured 24 is the highest reasonable value:

		// also limit how many peers we want to propose with respect to "peer synergy" (this
		// value can't be too high for performance reasons, 12 seems like a good middle-ground)
		const peersToProposeMaxWithSynergy = 12
		peersToProposeCnt = min(peersToProposeCnt, peersToProposeMaxWithSynergy)