feat: clean up p2p & implement missing peering functionality

[zivkovicmilos]

Description

Closes #2308

There is a lot happening in this PR, so don't be discouraged by the files changed.
I'll outline the way the PR should be reviewed, and give you pointers along the way.

What this PR set out to do

This PR initially set out to implement peer discovery in the TM2 p2p module -- that's it, basically.

The change was meant to be minimal, and not involve larger changes.
After spending more time than I'd like to admit in the p2p codebase, I've come to a couple of realizations:

• the code is insanely complex for what it needs to be doing.
• there are premature "optimizations" on every corner, with no real reason for them.
• the unit tests in the p2p package are sketchy to say the least, and not convincing at all that we were covering actual functionality we needed to cover.
• there are random disconnection issues with larger clusters.

All of these are temporarily fine, and not blocking us.

But the pattern was there -- to add peer discovery, it would require continuing the same pattern of code gymnastics present in the p2p module for the last 5+ years.

I took this opportunity, ahead of the mainnet launch, to fill out a few checkboxes:

• simplify the code, trim everything that's excess. This is in line with our project PHILOSOPHY.md.
• create a safety net in the form of integration tests, and unit tests, that run instantly, and give us the confidence stuff actually works.
• make it easy peasy for us to debug future p2p problems, when they arise (we already keep seeing them on existing testnets, like test4 and test5.

I wanted to implement all of this, without breaking any existing TM2 functionality that relies on the p2p module, or introducing additional complexities.

What this PR actually accomplished

I'm proud to say that this PR brings more than a few bells and whistles to the table, in terms of TM2 improvements:

• greatly simplified and faster Switch and Transport implementations. No more gazillion redundant checks, or expensive lookups, or convoluted APIs.
• a unit testing suite we can be proud of, and have confidence in. I rewrote the entire testing suite for the module, because the old implementation had severe limitations on mock-ability.
• peer discovery that works, and is not invasive. Goodbye random network pockets, and hanging nodes!
• many bugs, and potential issues squashed and erased. Regressions added, and passing.

For the sake of not making groundbreaking changes ahead of mainnet, I didn't touch a few things, and this will be evident from the code:

• I didn't touch the conn package, or how the multiplex connections are established and maintained (or the STS implementation) -- this would be too much, and require an exponential amount of time to get right.
• the Peer abstraction is still the same, and TM2 modules interact with the peers in the same way as before (directly, as Reactors). I've outlined the issues with this in the README, so check it out.

In retrospect, I should've limited the scope of this PR by a lot. At the time I was at the mid-way point, I committed fully to leaving this module in a better state than I found it in, rather than leave additional tech debt for future cleanup.

The other primary goal of this PR is to scope out changes needed to upgrade the networking layer implementation into utilizing a stack like libp2p. I am happy to say that we have 0 limitations in terms of p2p functionality to make the switch. We're just bound by time. This upgrade is scheduled for after the mainnet MVP launch 🤞

How do I review this PR?

There is no point in looking at the older implementation, and trying to figure out the changes from there.

There are just too many, and it can get overwhelming quickly.

Instead, as the first step -- read the new p2p README. It outlines how the p2p module works on a core level, and highlights current challenges.

After the README, open the p2p package in tm2/pkg/p2p, and start looking at the implementation from there. Leave comments on things that are unclear, or can be improved. I'll try to answer and give as much context as I can.

Contributors' checklist...

• Added new tests, or not needed, or not feasible
• Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory
• Updated the official documentation or not needed
• No breaking changes were made, or a BREAKING CHANGE: xxx message was included in the description
• Added references to related issues and PRs
• Provided any useful hints for running manual tests
• Added new benchmarks to generated graphs, if any. More info here.

[codecov]

Codecov Report

Attention: Patch coverage is 78.26633% with 346 lines in your changes missing coverage. Please review.

Project coverage is 63.69%. Comparing base (a1a7cb3) to head (7d64ff9).

Files with missing lines	Patch %	Lines
tm2/pkg/p2p/mock/peer.go	58.91%	72 Missing and 4 partials ⚠️
tm2/pkg/p2p/switch.go	81.68%	45 Missing and 5 partials ⚠️
tm2/pkg/p2p/discovery/discovery.go	78.57%	24 Missing and 6 partials ⚠️
tm2/pkg/p2p/types/netaddress.go	69.23%	26 Missing and 2 partials ⚠️
tm2/pkg/p2p/types/key.go	53.57%	22 Missing and 4 partials ⚠️
tm2/pkg/bft/node/node.go	73.41%	14 Missing and 7 partials ⚠️
tm2/pkg/p2p/transport.go	90.00%	12 Missing and 6 partials ⚠️
tm2/pkg/internal/p2p/p2p.go	89.75%	17 Missing ⚠️
tm2/pkg/p2p/set.go	82.19%	10 Missing and 3 partials ⚠️
tm2/pkg/p2p/dial/dial.go	75.67%	9 Missing ⚠️
... and 16 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2852      +/-   ##
==========================================
- Coverage   63.79%   63.69%   -0.11%     
==========================================
  Files         549      578      +29     
  Lines       78819    80176    +1357     
==========================================
+ Hits        50281    51065     +784     
- Misses      25146    25724     +578     
+ Partials     3392     3387       -5     

Flag	Coverage Δ
contribs/gnodev	61.16% <ø> (ø)
contribs/gnofaucet	14.82% <ø> (ø)
gnovm	67.93% <ø> (ø)
misc/genstd	79.72% <ø> (ø)
tm2	63.08% <78.18%> (+0.74%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[moul]

Unsigned may be better than signed if we don't need to use patterns like -1. However, since we're constrained by the OS limits, a size of 64 may be excessive.

[zivkovicmilos]

There is a leftover issue with this PR that I'm currently investigating, which is a hanging test somewhere in TM2.

I will resolve this before opening the PR for reviews 🙏