Skip to content
This repository has been archived by the owner on Jul 1, 2021. It is now read-only.

Implement eth/64 protocol #1543

Merged
merged 5 commits into from
Feb 24, 2020
Merged

Conversation

cburgdorf
Copy link
Contributor

@cburgdorf cburgdorf commented Feb 10, 2020

What was wrong?

Trinity is currently on eth/63 whereas Geth has implemented eth/64 and eth/65 already. We are pushing for the inclusion of request ids as eth/66.

Before we add an implementation for the proposed eth/66 protocol, we should first implement eth/64 and eth/65.

How was it fixed?

This implements the eth/64 protocol according to EIP 2124. The implementation follows the general principle of how LESV1, LESV2 are implemented side by side.

This also adds a tiny cleanup to combine three different places with the same if/else construct to choose the concrete API version into a single helper function.

Real life testing also demonstrates we are having eth/64 peers alongside eth/63 peers in the pool.

Adding ETHPeer (eth, 64) <Session <Node(0x089581@45.40.132.135)> 1da28744-11a1-4ce4-82e3-c84449cd41df> to pool
Adding ETHPeer (eth, 63) <Session <Node(0x5629f4@54.39.133.125)> 656b4238-5afd-4df8-8f29-515022183c0b> to pool

To-Do

  • Clean up commit history
  • Update tests
  • Fill in PR description
  • Cleanup code
  • Do the actual ForkID check and disconnect

Cute Animal Picture

put a cute animal picture link inside the parentheses

Copy link
Member

@pipermerriam pipermerriam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you can show that ETH/64 is supported our-of-the-box by the majority of network nodes (I don't know whether it actually is...) then we can drop ETH/63

@cburgdorf
Copy link
Contributor Author

If you can show that ETH/64 is supported our-of-the-box by the majority of network nodes (I don't know whether it actually is...) then we can drop ETH/63

Yeah, I thought about that too but ended up abandoning the thought because:

  1. Geth still supports eth/63
  2. It's a good code exercise to support two different handshakes especially when we already remove LES

@cburgdorf cburgdorf force-pushed the christoph/feat/eth64 branch 13 times, most recently from 65bccfc to dcf264b Compare February 14, 2020 13:52
@cburgdorf
Copy link
Contributor Author

Alright, this should be ready for review.

@cburgdorf
Copy link
Contributor Author

I have reassigned this to you @gsalgado because I think @pipermerriam is very busy right now.

Copy link
Contributor

@gsalgado gsalgado left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a few comments/suggestions

return chain


@pytest.fixture(params=(ETHV63PeerPairFactory, ETHPeerPairFactory))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, this causes all fixtures that use this one to be parameterized as well, I guess?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct! I tried to keep the tests structure similar to the LESV1 vs LESV2 tests that use the same pattern?

)
from trinity.tools.factories import (
BlockHashFactory,
ChainContextFactory,
ETHPeerPairFactory,
)
from trinity.tools.factories.eth.proto import ETHV63PeerPairFactory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this factory not imported from the same place as ETHPeerPairFactory?

Copy link
Contributor Author

@cburgdorf cburgdorf Feb 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is me testing out PyCharm Professional and relying on its auto import feature 😅 I'll fix that.

self.protocol.send(StatusV63(payload))


class ETHAPI(BaseETHAPI):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you chose not to include the version in the class name here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only put in the version numbers in classes that aren't on the latest version.

E.g.

ETHV63API
ETHProtocolV63
ETHV63PeerFactory

vs

ETHAPI
ETHProtocol
ETHPeerFactory

raise WrongForkIDFailure(
f"{multiplexer.remote} network "
f"({receipt.handshake_params.fork_id}) is incompatible to ours "
f"({self.handshake_params.fork_id}), disconnecting"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe include the actual exception msg here as well, as it would give extra details about where exactly the incompatibility lies.

)
except BaseForkIDValidationError:
raise WrongForkIDFailure(
f"{multiplexer.remote} network "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/network/forkid?

@cburgdorf cburgdorf force-pushed the christoph/feat/eth64 branch from dcf264b to 0a5a9f3 Compare February 20, 2020 09:16
@cburgdorf
Copy link
Contributor Author

cburgdorf commented Feb 20, 2020

Just ran a last live test and had this error blasting out.

ERROR  2020-02-20 10:20:24,392           ETHPeerPool  unexpected error during peer connection
Traceback (most recent call last):
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/commands.py", line 131, in decode
    payload = cls.serialization_codec.decode(payload_data)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/commands.py", line 64, in decode
    rlp.decode(data, strict=self.decode_strict, sedes=self.sedes, recursive_cache=True)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/rlp/codec.py", line 237, in decode
    obj = sedes.deserialize(item, **kwargs)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/eth_utils/functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/rlp/sedes/lists.py", line 89, in deserialize
    serial)
rlp.exceptions.ListDeserializationError: Deserializing list length (5) does not match sedes (6)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cburgdorf/Documents/hacking/ef/trinity/trinity/protocol/common/peer.py", line 206, in maybe_connect_more_peers
    for backend in self.peer_backends
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 209, in _add_peers_from_backend
    await self.connect_to_nodes(iter(candidates))
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 401, in connect_to_nodes
    loop=self.get_event_loop(),
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 437, in connect_to_node
    peer = await self.connect(node)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 347, in connect
    timeout=HANDSHAKE_TIMEOUT,
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 20, in wait
    return await self.wait_first(awaitable, token=token, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 43, in wait_first
    return await token_chain.cancellable_wait(*awaitables, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/cancel_token/token.py", line 163, in cancellable_wait
    await future
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer.py", line 492, in handshake
    token=self.cancel_token
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/handshake.py", line 323, in dial_out
    token=token,
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/handshake.py", line 288, in negotiate_protocol_handshakes
    in zip(selected_handshakers, selected_protocols)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/async_generator/_util.py", line 53, in __aexit__
    await self._agen.athrow(type, value, traceback)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 381, in multiplex
    fut.result()
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 404, in _do_multiplexing
    await self._handle_commands(msg_stream, stop)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 421, in _handle_commands
    async for protocol, cmd in msg_stream:
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 67, in wait_iter
    timeout=timeout,
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 20, in wait
    return await self.wait_first(awaitable, token=token, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 43, in wait_first
    return await token_chain.cancellable_wait(*awaitables, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/cancel_token/token.py", line 178, in cancellable_wait
    return done.pop().result()
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 87, in stream_transport_messages
    cmd = command_type.decode(msg, msg_proto.snappy_support)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/commands.py", line 136, in decode
    ) from err
rlp.exceptions.DeserializationError: DeserializationError for <class 'trinity.protocol.eth.commands.Status'>

I had live tested this before with a mix of eth/64 and eth/63 peers and had no errors. Need to check where this is coming from. It looks like someone is sending us an eth/63Status message where we expect an eth/64 one.

So I need to hold of merging this until I figured out what's wrong here.

@cburgdorf
Copy link
Contributor Author

@gsalgado Mind taking a look at the most recent commit that I added? This deals with the DeserializationError. From all I can tell, this has nothing to do with this PR per se. I think what happens is that we hit this error if we try to do a handshake with a client that advertises eth/64 but then actually responds with the wrong status message. This is now treated as a MalformedMessage.

try:
cmd = command_type.decode(msg, msg_proto.snappy_support)
except rlp.exceptions.DeserializationError:
raise MalformedMessage(f"Failed to decode {msg} for {command_type}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use raise MalformedMessage(...) from err, to include the original error?

@cburgdorf cburgdorf force-pushed the christoph/feat/eth64 branch from 732c53f to 0f1ecd0 Compare February 24, 2020 09:55
@cburgdorf cburgdorf merged commit f2d1bb4 into ethereum:master Feb 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants