Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework the transport upgrade API. #1240

Merged
merged 5 commits into from
Sep 10, 2019
Merged

Conversation

romanb
Copy link
Contributor

@romanb romanb commented Sep 5, 2019

This is a proposal for #953.

Motivation

Although transport upgrades must follow a specific pattern in order for the resulting transport to be usable with a Network or Swarm, that pattern is currently not well reflected in the transport upgrade API. Rather, transport upgrades are somewhat laborious and involve non-trivial code duplication.

Overview

This PR introduces a transport::upgrade::Builder that is obtained from Transport::upgrade. The Builder encodes the previously implicit rules for transport upgrades:

  1. Authentication upgrades must happen first.
  2. Any number of upgrades may follow.
  3. A multiplexer upgrade must happen last.

Since multiplexing is the last (regular) transport upgrade (because that upgrade yields a StreamMuxer which is no longer a AsyncRead / AsyncWrite resource, which the upgrade process is based on),
the upgrade starts with Transport::upgrade and ends with Builder::multiplex, which drops back down to the Transport, providing a fluent API.

Contract for authenticators and multiplexers

Authentication and multiplexer upgrades must adhere to a minimal contract w.r.t their outputs:

  1. An authentication upgrade is given an (async) I/O resource C and must produce a pair (I, D) where I: ConnectionInfo and D is a new (async) I/O resource D.

  2. A multiplexer upgrade is given an (async) I/O resource C and must produce a M: StreamMuxer.

To that end, two changes to the secio and noise protocols have been made:

  1. The secio upgrade now outputs a pair of (PeerId, SecioOutput). The former implements ConnectionInfo and the latter AsyncRead / AsyncWrite, fulfilling the Builder contract.

  2. A new NoiseAuthenticated upgrade has been added that wraps around any noise upgrade (i.e. NoiseConfig) and has an output of (PeerId, NoiseOutput), i.e. it checks if the RemoteIdentity from the handshake output is an IdentityKey, failing if that is not the case. This is the standard upgrade procedure one wants for integrating noise with libp2p-core/swarm.

Examples

A typical example before/after:

Before

   let transport = libp2p_tcp::TcpConfig::new()
            .with_upgrade(libp2p_secio::SecioConfig::new(local_key))
            .and_then(move |out, endpoint| {
                let peer_id = out.remote_key.into_peer_id();
                let peer_id2 = peer_id.clone();
                let upgrade = libp2p_mplex::MplexConfig::default()
                    .map_outbound(move |muxer| (peer_id, muxer))
                    .map_inbound(move |muxer| (peer_id2, muxer));
                upgrade::apply(out.stream, upgrade, endpoint)
            });

After

  let transport = libp2p_tcp::TcpConfig::new()
            .upgrade()
            .authenticate(libp2p_secio::SecioConfig::new(local_key))
            .multiplex(libp2p_mplex::MplexConfig::new());

Fore more examples, see the diff, which removes all of the related TODOs.

Since Transport::with_upgrade is removed and transport::upgrade::Upgrade changed to fit into the Builder pipeline, these changes are not fully backward-compatible, but adapting existing code should be easy.

Roman S. Borschel added 2 commits September 5, 2019 22:56
ALthough transport upgrades must follow a specific pattern
in order fot the resulting transport to be usable with a
`Network` or `Swarm`, that pattern is currently not well
reflected in the transport upgrade API. Rather, transport
upgrades are rather laborious and involve non-trivial code
duplication.

This commit introduces a `transport::upgrade::Builder` that is
obtained from `Transport::upgrade`. The `Builder` encodes the
previously implicit rules for transport upgrades:

  1. Authentication upgrades must happen first.
  2. Any number of upgrades may follow.
  3. A multiplexer upgrade must happen last.

Since multiplexing is the last (regular) transport upgrade (because
that upgrade yields a `StreamMuxer` which is no longer a `AsyncRead`
/ `AsyncWrite` resource, which the upgrade process is based on),
the upgrade starts with `Transport::upgrade` and ends with
`Builder::multiplex`, which drops back down to the `Transport`,
providing a fluent API.

Authentication and multiplexer upgrades must furthermore adhere
to a minimal contract w.r.t their outputs:

  1. An authentication upgrade is given an (async) I/O resource `C`
     and must produce a pair `(I, D)` where `I: ConnectionInfo` and
     `D` is a new (async) I/O resource `D`.
  2. A multiplexer upgrade is given an (async) I/O resource `C`
     and must produce a `M: StreamMuxer`.

To that end, two changes to the `secio` and `noise` protocols have been
made:

  1. The `secio` upgrade now outputs a pair of `(PeerId, SecioOutput)`.
     The former implements `ConnectionInfo` and the latter `AsyncRead` /
     `AsyncWrite`, fulfilling the `Builder` contract.

  2. A new `NoiseAuthenticated` upgrade has been added that wraps around
     any noise upgrade (i.e. `NoiseConfig`) and has an output of
     `(PeerId, NoiseOutput)`, i.e. it checks if the `RemoteIdentity` from
     the handshake output is an `IdentityKey`, failing if that is not the
     case. This is the standard upgrade procedure one wants for integrating
     noise with libp2p-core/swarm.
@romanb romanb marked this pull request as ready for review September 5, 2019 21:13
@romanb romanb mentioned this pull request Sep 9, 2019
2 tasks
Copy link
Contributor

@twittner twittner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@tomaka tomaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit

@@ -0,0 +1,109 @@

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs the license.

@romanb romanb merged commit 8c11926 into libp2p:master Sep 10, 2019
@romanb romanb deleted the upgrade-builder branch September 10, 2019 13:42
ackintosh added a commit to ackintosh/rust-libp2p that referenced this pull request Sep 20, 2019
tomaka pushed a commit that referenced this pull request Oct 11, 2019
* WIP

* plaintext/2.0.0

* Refactor protobuf related issues to compatible with the spec

* Rename: new PlainTextConfig -> PlainText2Config

* Keep plaintext/1.0.0 as PlainText1Config

* Config contains pubkey

* Rename: proposition -> exchange

* Add PeerId to Exchange

* Check the validity of the remote's `Exchange`

* Tweak

* Delete unused import

* Add debug log

* Delete unused field: public_key_encoded

* Delete unused field: local

* Delete unused field: exchange_bytes

* The inner instance should not be public

* identity::Publickey::Rsa is not available on wasm

* Delete PeerId from Config as it should be generated from the pubkey

* Catch up for #1240

* Tweak

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update protocols/plaintext/src/handshake.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>

* Rename: pubkey -> local_public_key

* Delete unused error

* Rename: PeerIdValidationFailed -> InvalidPeerId

* Fix: HandShake -> Handshake

* Use bytes insteadof Publickey to avoid code duplication

* Replace with ProtobufError

* Merge HandshakeContext<()> into HandshakeContext<Local>

* Improve the peer ID validation to simplify the handshake

* Propagate Remote to allow extracting the PeerId from the Remote

* Collapse the same kind of errors into the variant
tomaka added a commit that referenced this pull request Oct 28, 2019
* Configurable multistream-select protocol. Add V1Lazy variant. (#1245)

Make the multistream-select protocol (version) configurable
on transport upgrades as well as for individual substreams.

Add a "lazy" variant of multistream-select 1.0 that delays
sending of negotiation protocol frames as much as possible
but is only safe to use under additional assumptions that
go beyond what is required by the multistream-select v1
specification.

* Improve the code readability of the chat example (#1253)

* Add bridged chats (#1252)

* Try fix CI (#1261)

* Print Rust version on CI

* Don't print where not appropriate

* Change caching strategy

* Remove win32 build

* Remove win32 from list

* Update libsecp256k1 dep to 0.3.0 (#1258)

* Update libsecp256k1 dep to 0.3.0

* Sign now cannot fail

* Upgrade url and percent-encoding deps to 2.1.0 (#1267)

* Upgrade percent-encoding dep to 2.1.0

* Upgrade url dep to 2.1.0

* Revert CIPHERS set to null (#1273)

* Update dependency versions (#1265)

* Update versions of many dependencies

* Bump version of rand

* Updates for changed APIs in rand, ring, and webpki

* Replace references to `snow::Session`

`Session` no longer exists in `snow` but the replacement is two structs `HandshakeState` and `TransportState`
Something will have to be done to harmonize `NoiseOutput.session`

* Add precise type for UnparsedPublicKey

* Update data structures/functions to match new snow's API

* Delete diff.diff

Remove accidentally committed diff file

* Remove commented lines in identity/rsa.rs

* Bump libsecp256k1 to 0.3.1

* Implement /plaintext/2.0.0 (#1236)

* WIP

* plaintext/2.0.0

* Refactor protobuf related issues to compatible with the spec

* Rename: new PlainTextConfig -> PlainText2Config

* Keep plaintext/1.0.0 as PlainText1Config

* Config contains pubkey

* Rename: proposition -> exchange

* Add PeerId to Exchange

* Check the validity of the remote's `Exchange`

* Tweak

* Delete unused import

* Add debug log

* Delete unused field: public_key_encoded

* Delete unused field: local

* Delete unused field: exchange_bytes

* The inner instance should not be public

* identity::Publickey::Rsa is not available on wasm

* Delete PeerId from Config as it should be generated from the pubkey

* Catch up for #1240

* Tweak

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update protocols/plaintext/src/handshake.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>

* Update protocols/plaintext/src/error.rs

Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>

* Rename: pubkey -> local_public_key

* Delete unused error

* Rename: PeerIdValidationFailed -> InvalidPeerId

* Fix: HandShake -> Handshake

* Use bytes insteadof Publickey to avoid code duplication

* Replace with ProtobufError

* Merge HandshakeContext<()> into HandshakeContext<Local>

* Improve the peer ID validation to simplify the handshake

* Propagate Remote to allow extracting the PeerId from the Remote

* Collapse the same kind of errors into the variant

* [noise]: `sodiumoxide 0.2.5` (#1276)

Fixes rustsec/advisory-db#192

* examples/ipfs-kad.rs: Remove outdated reference to `without_init` (#1280)

* CircleCI Test Fix (#1282)

* Disabling "Docker Layer Caching" because it breaks one of the circleci checks

* Bump to trigger CircleCI build

* unbump

* zeroize: Upgrade to v1.0 (#1284)

v1.0 final release is out. Release notes:

iqlusioninc/crates#279

* *: Consolidate protobuf scripts and update to rust-protobuf 2.8.1 (#1275)

* *: Consolidate protobuf generation scripts

* *: Update to rust-protobuf 2.8.1

* *: Mark protobuf generated modules with '_proto'

* examples: Add distributed key value store (#1281)

* examples: Add distributed key value store

This commit adds a basic distributed key value store supporting GET and
PUT commands using Kademlia and mDNS.

* examples/distributed-key-value-store: Fix typo

* Simple Warning Cleanup (#1278)

* Cleaning up warnings - removing unused `use`

* Cleaning up warnings - unused tuple value

* Cleaning up warnings - removing dead code

* Cleaning up warnings - fixing deprecated name

* Cleaning up warnings - removing dead code

* Revert "Cleaning up warnings - removing dead code"

This reverts commit f18a765.

* Enable the std feature of ring (#1289)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants