IBFT in parity?

[drandreaskrueger]

@Tbaut suggested in https://gitter.im/paritytech/parity-poa to post this here; quotes are from there. Please keep this issue open until there is an implementation.

port IBFT from go to rust ?

if you want IBFT, you need to use quorum.

Thanks.

In case we use IBFT for our next project (however not decided yet), we would like to still use parity - that has always been our Ethereum client of choice here at Electron.org.uk .

I suggest porting the go IBFT (see EIP-650) to rust, and to include it into official parity

Yes, we could do all this alone, but then we must maintain some idiosyncratic fork of parity - and then always having to catch up with the parity master, etc. (For example, the EWF parity client (with the "tobalaba chain") is ~a thousand commits behind. Not ideal.)

We actually do have a rust coder who could possibly (help to) port the existing IBFT go code to rust. But someone who already knows parity inside out would of course get faster results at how exactly to insert another consensus engine into the existing parity client.

Please discuss this in the parity team. Thanks!

geth doesn't support Aura, and Parity Ethereum doesn't support Clique

With that possible future port of IBFT from go to rust, there would then finally be a PoA which can work in all three clients (quorum, geth, parity). A good step forward, no?

[drandreaskrueger]

Helpful resources:

• EIP 650: Istanbul Byzantine Fault Tolerance, Jun 22, 2017
• interface discussion, Mar 26 - incl the/some code
• golang code: quorum --> consensus --> istanbul
• Istanbul BFT (IBFT), Nov 24, 2017, Zih-Ci Lin, AMIS developer
• Scaling Consensus for Enterprise: Explaining the IBFT Algorithm, PegaSys (ConsenSys), Jun 22
• IBFT is based on PBFT - Practical Byzantine Fault Tolerance, Miguel Castro & Barbara Liskov, 1999
• geth_quorum --ottoman is a public testnet (currently offline)
• DevconThree talk: 5 minutes video
• slides: Istanbul BFT - Yu-Te Lin, Software Engineer @ AMIS

[Tbaut]

Ping @rphmeier to comment on that :)

[rphmeier]

The on-chain voting algorithm of Clique and Istanbul will probably take some effort to code. Other than that, Istanbul will require the same or similar changes as #8888. We are currently collaborating with POA network to get these baseline changes in, with the intent of including HoneyBadgerBFT as a consensus engine in Parity-Ethereum.

Beyond that, it seems that there is a serious demand for these kinds of engines in Parity-Ethereum and we will gladly accept any external contribution to implement them, but they are also a significant undertaking and the resources of the core development team are currently elsewhere.

[drandreaskrueger]

Thanks a lot! Good to know.

Anyone else? Other opinions?

will probably take some effort to code.

Yes.
There is a working implementation in golang; porting that 1:1 to rust should need less time than developing it from scratch, no?
I am now also contacting the original authors from AMIS, perhaps they can help too?

will require the same or similar changes as #8888
currently collaborating with POA network to get these baseline changes in

Ah, good, very good.
Approx ideas about an ETA?
Who to contact, or invite to this issue here?

HoneyBadgerBFT

Prefered over IBFT? Why?

Is there also a geth implementation of HoneyBadgerBFT ?

Is there any other PoA algorithm which might (soon) run on both platforms, geth and parity? (or: why not?)

gladly accept any external contribution to implement them

As mentioned, we are interested.
(Because our only other alternative would be to drop parity now, and instead from now on work with geth; but we have developed some tools which are parity specific ... so it's a question of porting those to geth, or helping to port IBFT to parity ...)

However, to plug the then new IBFT rust code into the existing parity codebase ... is easier with support from the parity core team.

How modular is parity, in terms of swapping out the consensus mechanism?

currently elsewhere

When might that change?
What would be needed?

[drandreaskrueger]

idea: https://gitcoin.co/explorer

[drandreaskrueger]

Aura seems to be slow at least for smart contract transactions - perhaps that could be another motivation to consider IBFT for parity?

[5chdn]

Yes, we could do all this alone, but then we must maintain some idiosyncratic fork of parity

Just go ahead, do it and submit a PR. Once it's in our code-base you don't have to maintain a fork, only the IBFT parts.

[rphmeier]

Worth noting that github.com/paritytech/rhododendron is quite close already to IBFT (only a few fairly small changes would be necessary to transform it into IBFT agreement). You'd still have to write the integration into Parity and validator-set switching mechanism.

[afck]

Approx ideas about an ETA?
Who to contact, or invite to this issue here?

Sorry, I only just found this issue. Our (network-agnostic, not Parity-specific) Honey Badger BFT implementation is here. Feel free to invite me and/or the other developers on related issues! I'm sure there's a lot of overlap with what we're trying to do, and it would be great to collaborate!

In terms of Parity integration, we're still exploring what the best path to integration is. One of the complications is that Honey Badger BFT relies heavily on low-latency high-bandwidth single-recipient (not multicast) messages between the validators.

Prefered over IBFT? Why?

To quote the Honey Badger BFT paper: "(weakly) synchronous protocol such as PBFT […] rely critically on network timing assumptions, and only guarantee liveness when the network behaves as expected. We argue these protocols are ill-suited for this deployment scenario." (Referring to cryptocurrencies.)
Honey Badger BFT is fully asynchronous, censorship-resistant, very efficient in message size, and leaderless. We're hoping that makes it very performant and resilient, but of course it's hard to tell how it plays out in practice until we can actually try it out in a live Parity-based network, and compare it to other algorithms.

Is there also a geth implementation of HoneyBadgerBFT ?
Is there any other PoA algorithm which might (soon) run on both platforms, geth and parity? (or: why not?)

For now we are focusing on integrating hbbft with Parity. In the future, getting it into geth, too, or adding more algorithms would be awesome, of course.

[drandreaskrueger]

Sorry for the longer silence. An update on this ^

Just go ahead, do it and submit a PR.

Very probably won't happen. Sorry.

paritytech/rhododendron is quite close already to IBFT ... only a few fairly small changes

Interesting. Would it hurt much to build those missing pieces?
If no manpower, perhaps incentivize via https://explorer.bounties.network/ ?

Even if you think IBFT is not yet the best possible solution to the problem, to have a first PoA algo that exists on both clients ... wouldn't that be good?

[drandreaskrueger]

(network-agnostic, not Parity-specific) Honey Badger BFT implementation

cool. I assume it can more easily be included into parity (because rust)? What about porting it to go?

Perhaps same approach?

Once HBBFT exists in those two languages, there is a real chance for finally both clients being able to run the same PoA algorithm, no?

Honey Badger BFT is fully asynchronous, censorship-resistant, very efficient in message size, and leaderless.

Sounds very good.

Any expectations how it might perform compared to (Aura, Clique, IBFT)?

What about an HBBFT EIP like the IBFT EIP-650.

[afck]

I agree, having compatible HBBFT implementations in both clients would be awesome!
The Honey Badger is a pretty complex beast, though, and it doesn't fit as readily into the "propose blocks and agree on them" scheme as most of the alternative mechanisms, so it's probably a lot more complicated to port and integrate.

That's why we're focusing only on the Rust/Parity version for now.

Any expectations how it might perform compared to (Aura, Clique, IBFT)?

We'll only know for sure once we've tried it out in the wild, of course.
A key difference with e.g. Aura, Ethash and Clique (I think) is that it's fully asynchronous: these all configure some timing/block rate parameters, whereas Honey Badger has no such settings at all. It simply always goes as fast as the network allows.
In comparison with PBFT/IBFT, it makes the promise that it's much harder to slow down. Problems (crashes, slow connection, DoS attacks) with the current leader/proposer can potentially cause delays, which Honey Badger should be mostly immune to.
We also hope that it has another speed advantage due to its message complexity: while it probably has more message rounds per epoch, its messages are much smaller than a block.

(Edit: With our simulation example, you can get rough estimates of how the algorithm will perform, given particular network latency, bandwidth, CPU speed, block size, etc.)

[drandreaskrueger]

simulation example

Great. No chance to try it until after devcon, but already linked to it here.

[adria0]

Closing issue due to its stale state.