[WIP] P2P: initial I2P support #2932

chisa0a · 2019-06-29T21:09:10Z

Adds initial support for I2P P2P connections. Includes @ignopeverell's initial work supporting I2P, finishes a large amount of the remaining work for E2E I2P connections, and rebases onto the milestone/2.x.x branch.

Supports connecting to an I2P seeding peer via the List seeding option in the config. Other config peer lists (peers_allow, peers_deny) also support I2P peers. Further testing is needed to see how other peer lists handle I2P peers.

Due to the size of this PR, will split remaining work into separate PRs building on top of this one.

Remaining work:

seeding I2P peers (currently I2P seeder only seeds TCP peers)
connecting exclusively through I2P
wallet I2P connections
miner I2P connections
creating bootstrap list of I2P seeders
reporting connected I2P peers in TUI
(maybe) installing i2pd automatically (if not found locally)

Changes have been tested on Mainnet and Floonet, with successful I2P connections on both. Currently, I2P peer connections have to be manually verified through i2pd user interface.

Seeding peer was able to sync exclusively from I2P seeder, get a TCP peer list from I2P seeder, and sync from TCP peers.

I've added a wiki entry for initial setup to aid with testing, and as WIP documentation for connecting Grin nodes over I2P (thanks for the suggestion @Kargakis).

Includes unit tests for de/serializing I2P peers from/to the config. Tests for creating I2P streams require a running I2P node that supports SAM connections.

Adds necessary configuration bits to enable I2P in various modes, unused for now but will soon. Also add peer address wrappers for I2P so those addresses can be understood and exchanged between peers. Doesn't compile yet as serde needs to be supported in i2p-rs first.

Convert use of panic! to return result + error. Adds PeerAddrType enum for converting address type to/from raw u8.

Adds I2P to fullnode capabilities, modifies tests accordingly. Other rustfmt changes.

Shutdown I2P streams gracefully on stop_state.

Includes `i2p_mode` and I2P peer examples in the default config file.

Adds implementation to start i2pd automatically when I2P support enabled.

Fixes references for non-`Copy` `PeerAddr`, other minor fixups for return types, and removing unneeded functions.

chisa0a · 2019-06-29T21:24:26Z

macosx test failed, but looks unrelated to the changes in this PR:

thread 'hard_forks' panicked at 'assertion failed: valid_header_version(YEAR_HEIGHT / 2 - 1, HeaderVersion::new(1))', core/tests/consensus.rs:628:3
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:208
   4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:474
   5: alloc::alloc::dealloc
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panicking.rs:408
   6: consensus::hard_forks
             at core/tests/consensus.rs:628
   7: consensus::hard_forks::{{closure}}
             at core/tests/consensus.rs:620
   8: core::ops::function::FnOnce::call_once
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libcore/ops/function.rs:231
   9: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/liballoc/boxed.rs:702
  10: panic_unwind::dwarf::eh::read_encoded_pointer
             at src/libpanic_unwind/lib.rs:87
  11: test::run_test::run_test_inner::{{closure}}
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panicking.rs:272
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panic.rs:388
             at src/libtest/lib.rs:1468

hashmap

Great works, some minor comments

p2p/Cargo.toml

p2p/src/handshake.rs

p2p/src/peers.rs

servers/src/grin/seed.rs

servers/src/grin/server.rs

hashmap · 2019-07-05T14:00:36Z

@chisa0a if you don't mind I can remove extra clone calls

chisa0a · 2019-07-05T20:57:48Z

@chisa0a if you don't mind I can remove extra clone calls

Sounds good to me. I can also take care of them when working on the other refactors, either way.

Many thanks for the review!

0xmichalis · 2019-07-09T11:58:49Z

config/src/comments.rs

+#mode = \"Enabled\"
+
+#[server.p2p_config.i2p_mode.i2p_config]
+#autostart = true/false  - start I2P automatically


More admin-friendly config:

# Start I2P automatically # autostart = true # Exclusively connect through I2P, or also use TCP/IP # exclusive = true

I missed this before the most recent push of updates. Can add with next round of changes.

We are interested only in behaviour of Stream, so it sounds right to model it as a trait. It allows also reuse implementation of existing traits (Read, Write) as is without wrapping. It simplifies client code because it can work directly with TcpStream and I2pStream. At the same time it brings some complexity - one more type parameter for some functions and some boilerplate code to implement trait for references.

Represent Stream as a trait

Convert use of panic! to return result + error. Adds PeerAddrType enum for converting address type to/from raw u8.

Adds I2P to fullnode capabilities, modifies tests accordingly. Other rustfmt changes.

Shutdown I2P streams gracefully on stop_state.

Includes `i2p_mode` and I2P peer examples in the default config file.

Adds implementation to start i2pd automatically when I2P support enabled.

Fixes references for non-`Copy` `PeerAddr`, other minor fixups for return types, and removing unneeded functions.

We are interested only in behaviour of Stream, so it sounds right to model it as a trait. It allows also reuse implementation of existing traits (Read, Write) as is without wrapping. It simplifies client code because it can work directly with TcpStream and I2pStream. At the same time it brings some complexity - one more type parameter for some functions and some boilerplate code to implement trait for references.

Adds `set_read_timeout` and `set_write_timeout` handlers to the Stream trait. Requires updates in i2p-rs for calling the functions on I2pStream.

Adds I2P thread handle to deal with any potential errors when joining on shutdown.

Various small fixups for rebase onto master at commit ea1d14d.

Changes the i2p-rs crate to the fork hosted on the mimblewimble org repo.

DavidBurkett · 2019-08-25T04:43:42Z

servers/src/grin/i2p.rs

+	}
+}
+
+fn start_i2pd() {


I don't see us stopping i2pd. Is that intentional?

quentinlesceller · 2020-08-17T17:39:39Z

Closing due to lack of activity.

ignopeverell and others added 20 commits June 27, 2019 22:25

Many reference fixes now that PeerAddr isn't Copy

2dd7ea1

Move i2p config to p2p, set capabilities

a9738c5

I2P connection init and addr/key handling

0081cf6

Starting listener impl and The Great Wrapping

3696b16

Listener impl more or less done, but untested

22b2432

rustfm

f1f3849

i2p client connection impl; fix existing p2p tests

396efc1

rustfmt

7d01407

P2P: remove panic!, add PeerAddrType enum

f6f0484

Convert use of panic! to return result + error. Adds PeerAddrType enum for converting address type to/from raw u8.

P2P: rustfmt + add I2P as fullnode capability

0e16fd1

Adds I2P to fullnode capabilities, modifies tests accordingly. Other rustfmt changes.

P2P: gracefully shutdown I2P listener

a2d823f

Shutdown I2P streams gracefully on stop_state.

P2P: rust-fmt

d2682cd

P2P: add serde decorator for PeerAddr enum

f420a9a

P2P: add serde decorators for I2pMode enum

77735f9

P2P: rust-fmt

828b8fd

Config: adds i2p_mode section + I2P peer examples

a1313b5

Includes `i2p_mode` and I2P peer examples in the default config file.

I2P: create keyfile directory if needed

63a2507

I2P: start i2pd by default

7264f0a

Adds implementation to start i2pd automatically when I2P support enabled.

P2P: various fixups for 2.x.x

f6cbcfd

Fixes references for non-`Copy` `PeerAddr`, other minor fixups for return types, and removing unneeded functions.

hashmap reviewed Jul 4, 2019

View reviewed changes

hashmap added 2 commits July 8, 2019 21:53

Remove extra clones

be57444

address review comments

98d5ffb

0xmichalis reviewed Jul 9, 2019

View reviewed changes

hashmap added 2 commits July 10, 2019 23:37

Merge pull request mimblewimble#5 from cyclefortytwo/stream-trait

f627e8e

Represent Stream as a trait

chisa0a force-pushed the i2p-2.x.x branch from f627e8e to 889239d Compare July 13, 2019 03:58

ignopeverell and others added 21 commits August 9, 2019 05:06

rustfm

31b49b0

i2p client connection impl; fix existing p2p tests

bc6b906

rustfmt

914ad0e

P2P: remove panic!, add PeerAddrType enum

f555e30

Convert use of panic! to return result + error. Adds PeerAddrType enum for converting address type to/from raw u8.

P2P: rustfmt + add I2P as fullnode capability

cfceb2b

Adds I2P to fullnode capabilities, modifies tests accordingly. Other rustfmt changes.

P2P: gracefully shutdown I2P listener

a693faa

Shutdown I2P streams gracefully on stop_state.

P2P: add serde decorator for PeerAddr enum

f3dfff9

P2P: add serde decorators for I2pMode enum

a1de0ff

P2P: rust-fmt

3012c2a

Config: adds i2p_mode section + I2P peer examples

23fc301

Includes `i2p_mode` and I2P peer examples in the default config file.

I2P: create keyfile directory if needed

fb4cd81

I2P: start i2pd by default

8fd318f

Adds implementation to start i2pd automatically when I2P support enabled.

P2P: various fixups for 2.x.x

8490ace

Fixes references for non-`Copy` `PeerAddr`, other minor fixups for return types, and removing unneeded functions.

Remove extra clones

b9564a7

address review comments

49cecb7

P2P: add set_read/write_timeout to Stream trait

3b1ce97

Adds `set_read_timeout` and `set_write_timeout` handlers to the Stream trait. Requires updates in i2p-rs for calling the functions on I2pStream.

P2P: add i2p_thread for error handling on shutdown

6c53ba1

Adds I2P thread handle to deal with any potential errors when joining on shutdown.

P2P: minor style + doc updates for PeerAddr

14b7b46

P2P: fixes for rebase onto ea1d14d

8e2f1a4

Various small fixups for rebase onto master at commit ea1d14d.

I2P: change i2p-rs to mimblewimble org repo

d65f77f

Changes the i2p-rs crate to the fork hosted on the mimblewimble org repo.

chisa0a force-pushed the i2p-2.x.x branch from bee4a64 to d65f77f Compare August 9, 2019 05:47

hashmap added 3 commits August 22, 2019 16:11

Merge to fix forced push

7cabd44

Make i2p config param optional

5bdae1e

Merge remote-tracking branch 'origin/master' into i2p-2.x.x

8628f80

DavidBurkett reviewed Aug 25, 2019

View reviewed changes

servers/src/grin/i2p.rs

}

}

fn start_i2pd() {

Copy link

Contributor

DavidBurkett Aug 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see us stopping i2pd. Is that intentional?

lehnberg mentioned this pull request Aug 27, 2019

Release Planning: Grin v2.1.0 mimblewimble/grin-pm#179

Closed

antiochp mentioned this pull request Sep 4, 2019

Online Transacting via TOR Hidden Services mimblewimble/grin-rfcs#24

Merged

quentinlesceller closed this Aug 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] P2P: initial I2P support #2932

[WIP] P2P: initial I2P support #2932

chisa0a commented Jun 29, 2019 •

edited

Loading

chisa0a commented Jun 29, 2019

hashmap left a comment •

edited

Loading

hashmap commented Jul 5, 2019

chisa0a commented Jul 5, 2019

0xmichalis Jul 9, 2019

chisa0a Jul 13, 2019

DavidBurkett Aug 25, 2019

quentinlesceller commented Aug 17, 2020

[WIP] P2P: initial I2P support #2932

[WIP] P2P: initial I2P support #2932

Conversation

chisa0a commented Jun 29, 2019 • edited Loading

chisa0a commented Jun 29, 2019

hashmap left a comment • edited Loading

Choose a reason for hiding this comment

hashmap commented Jul 5, 2019

chisa0a commented Jul 5, 2019

0xmichalis Jul 9, 2019

Choose a reason for hiding this comment

chisa0a Jul 13, 2019

Choose a reason for hiding this comment

DavidBurkett Aug 25, 2019

Choose a reason for hiding this comment

quentinlesceller commented Aug 17, 2020

chisa0a commented Jun 29, 2019 •

edited

Loading

hashmap left a comment •

edited

Loading