[WIP] Integrating libp2p into Lodestar

[Mikerah]

This is currently a work in progress and breaks tests.

I have started integrating libp2p into Lodestar. A simple libp2p bundle has been created and is subject to change as more components get built out.

Here's a current TODO list:

• Have a working basic server for managing peers

• (Optional) Change part of the service API

• Add RPC over libp2p to start integrating the ETH2.0 Wire Protocol

• Add BDD tests to make sure networking is fine.

[CLAassistant]

All committers have signed the CLA.

[GregTheGreek]

Just quickly glanced, we should be probably importing a libp2p library rather than a bunch of individual ones?

[Mikerah]

@GregTheGreek The way it works is that if you don't need any specialized transports, multiplexers, etc and you can use the ones provided by libp2p, you need to add them to the project individually. The one that might be optional is FloodSub but I still imported so that when GossipSub is done, I can just replace the import.

[wemeetagain]

very exciting, starting to take shape :)

[wemeetagain]

@GregTheGreek I think this is as it should be re: importing libraries for now. libp2p takes a veeery modular approach, and we need all of these libraries to weave everything together properly. In fact, we probably will be adding even more libp2p libraries in the future!

Its probably easier to start here, monorepo style, and once we've got things more solid, pull out into a separate repo/ repos, if it makes sense.

[GregTheGreek]

@wemeetagain Fair enough, I iguess we have a lot to trimfrom the package.json anyways.

[wemeetagain]

@Mikerah would you be willing to do minor cleanup and plan on merging as is?
It would probably be better to just have the code in master and iterate from there.
I think your TODOs could be reintroduced as issues for further PRs.

[Mikerah]

@wemeetagain I can do minor cleanups and separate the TODO as separate PRs. My main bottleneck right now is that all the tests break due to odd compilation issues. Before merging, I would like to confirm that this is just an issue on my end and not everybody else's.

[wemeetagain]

I think what you're seeing is a result of our check-types script.
Before running tests, we run the typescript compiler (without output) just for it to do some checks and catch any type errors.
It's very picky and will catch many small issues.
You'd probably want to just go through them, one by one.

If you're using an IDE (or vim w/ plugins like typescript-vim and YouCompleteMe), it may be much easier, because the errors will be called out in the editor and potentially can be auto-fixed.

[Mikerah]

The basics are in here and I will be opening subsequent PRs. @mpetrunic got the types working.

[codecov]

Codecov Report

Merging #154 into master will decrease coverage by 20.5%.
The diff coverage is 22.66%.

@@             Coverage Diff             @@
##           master     #154       +/-   ##
===========================================
- Coverage   52.79%   32.29%   -20.51%     
===========================================
  Files          85       47       -38     
  Lines        1305      737      -568     
  Branches      123       64       -59     
===========================================
- Hits          689      238      -451     
+ Misses        590      499       -91     
+ Partials       26        0       -26

[Mikerah]

The build is now passing. Awaiting approval...

Updates have been applied

[GregTheGreek]

Few small async based comments, but also theres a lot of callback hell going on, using promisify would really help

[Mikerah]

@GregTheGreek I think leaving the waterfall in createNode is fine. This function won't be changing much in the future. I also think that waterfall is more readable than promisify in the way it's currently coded. However, I found this issue that seems like it's a compromise between both ways.

[mpetrunic]

@GregTheGreek I think leaving the waterfall in createNode is fine. This function won't be changing much in the future. I also think that waterfall is more readable than promisify in the way it's currently coded. However, I found this issue that seems like it's a compromise between both ways.

I think the point is we don't wan't both async an promisify librarys since they both do same job

[wemeetagain]

It looks like createNode was pulled from floodsub.
In gossipsub, I recently promisifyed it, you can see it here: https://github.com/ChainSafe/gossipsub-js/blob/gossip_implementation/test/utils.js#L18

[Mikerah]

@wemeetagain createNode was pulled from one of the libp2p examples. They use that pretty much everywhere. I don't mind promisifying.

[wemeetagain]

You can remove the async dependency

Edit: you should take a look at the typescript checking too (yarn check-types), causing travis failure.