Default validator support #525

vyzo · 2023-02-28T20:39:38Z

Adds support for default validators and provides an implementation that validates the message seqno as a per peer nocne to prevent infinite propagation when the network diameter exceeds the seen cache span.
See #524
Implements the logic described in ipfs/kubo#9665

TBD:

unit test

MarcoPolo

One suggestion to the doc comment. Would be nice to have some tests here as well, but understood that this is urgent so we can defer to a future PR.

validation_builtin.go

stbrody · 2023-03-01T22:03:34Z

Is there a risk here if messages are delivered out of order? Like if a node receives message 5 before message 4 from the same peer, will that node then never forward message 4 on to its other peers? Does that risk some messages not being well propagated to the whole network?

vyzo · 2023-03-02T09:25:46Z

This can only happen if one message from the peer is dropped (because of a full queue) and the next message propagates before the dropped message arrives from other paths.

So in theory there is some risk, but this would be quite difficult to trigger.
Maybe with an extremely high sending rate from some peer.

Worth noting, but i am not convinced it is a problem.

vyzo · 2023-03-04T18:52:04Z

There is a simple way to alleviate concerns here: keep a bitvector (say 32 or 64 bits, as an integer) that encodes whether the last few messages have been delivered.

How does that sound?

stbrody · 2023-03-05T15:56:29Z

@vyzo I don't think I fully understand how the bitvector would work, but I think keeping a small cache of the last few seqnos and whether they've been delivered once before does make sense and would address the issue I brought up!

vyzo · 2023-03-05T16:00:22Z

Ok, i will implement something along these lines.

* feat: expire messages from the cache based on last seen time (libp2p#513) * feat: expire messages from the cache based on last seen time * chore: minor renaming * fix: messages should not be found after expiration * chore: editorial * fix: use new time cache strategy consistently * fix: default to old time cache and add todo for background gc * chore: update to go-libp2p v0.25 (libp2p#517) * Update to go-libp2p v0.25 * Use go 1.19 * chore: update go version and dependencies (libp2p#516) * fix(timecache): remove panic in first seen cache on Add (libp2p#522) * Refactor timecache implementations (libp2p#523) * reimplement timecache for sane and performant behaviour * remove seenMessagesMx, take advantage of new tc api * fix timecache tests * fix typo * store expiry, don't make life difficult * refactor common background sweep procedure for both impls * add godocs to TimeCache * Default validator support (libp2p#525) * add default validator support * add an implementation for basic seqno as nonce validation * missing return * the nonce belongs to the origin peer * add note about rust predicament * add seqno validator tests * minor test tweak, ensure at least 1ms before replay * Fix Memory Leak In New Timecache Implementations (libp2p#528) * fix bug * add for last seen cache * chore: Update .github/workflows/stale.yml [skip ci] * chore: Update .github/workflows/stale.yml [skip ci] * upgrades libp2p version * upgrades libp2p version --------- Co-authored-by: Mohsin Zaidi <2236875+smrz2001@users.noreply.github.com> Co-authored-by: Marco Munizaga <git@marcopolo.io> Co-authored-by: RichΛrd <info@richardramos.me> Co-authored-by: Hlib Kanunnikov <hlibwondertan@gmail.com> Co-authored-by: vyzo <vyzo@hackzen.org> Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: GitHub <noreply@github.com>

* feat: expire messages from the cache based on last seen time (libp2p#513) * feat: expire messages from the cache based on last seen time * chore: minor renaming * fix: messages should not be found after expiration * chore: editorial * fix: use new time cache strategy consistently * fix: default to old time cache and add todo for background gc * chore: update to go-libp2p v0.25 (libp2p#517) * Update to go-libp2p v0.25 * Use go 1.19 * chore: update go version and dependencies (libp2p#516) * fix(timecache): remove panic in first seen cache on Add (libp2p#522) * Refactor timecache implementations (libp2p#523) * reimplement timecache for sane and performant behaviour * remove seenMessagesMx, take advantage of new tc api * fix timecache tests * fix typo * store expiry, don't make life difficult * refactor common background sweep procedure for both impls * add godocs to TimeCache * Default validator support (libp2p#525) * add default validator support * add an implementation for basic seqno as nonce validation * missing return * the nonce belongs to the origin peer * add note about rust predicament * add seqno validator tests * minor test tweak, ensure at least 1ms before replay * Fix Memory Leak In New Timecache Implementations (libp2p#528) * fix bug * add for last seen cache * chore: Update .github/workflows/stale.yml [skip ci] * chore: Update .github/workflows/stale.yml [skip ci] * bump golang.org/x/net from 0.4.0 to 0.7.0 (libp2p#520) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](golang/net@v0.4.0...v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: topicscore params can't be set for dynamically subscribed topic (libp2p#540) * fix: topicscore params can't be set for a topic subscribed after gossipsub is initialized * chore:address review comments * Revert "fix: topicscore params can't be set for dynamically subscribed topic (libp2p#540)" (libp2p#541) This reverts commit aa5fd79. * remove usage of deprecated peerid.Pretty method (libp2p#542) * chore: update go-libp2p to v0.32 (libp2p#548) * chore: Update .github/workflows/stale.yml [skip ci] * make tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Mohsin Zaidi <2236875+smrz2001@users.noreply.github.com> Co-authored-by: Marco Munizaga <git@marcopolo.io> Co-authored-by: RichΛrd <info@richardramos.me> Co-authored-by: Hlib Kanunnikov <hlibwondertan@gmail.com> Co-authored-by: vyzo <vyzo@hackzen.org> Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: GitHub <noreply@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com> Co-authored-by: Sukun <sukunrt@gmail.com>

vyzo added 3 commits February 28, 2023 22:12

add default validator support

54e0d44

add an implementation for basic seqno as nonce validation

37d4656

missing return

c361b1d

vyzo requested a review from MarcoPolo February 28, 2023 20:39

vyzo mentioned this pull request Feb 28, 2023

Pubsub flood due to same message propagated multiple times ipfs/kubo#9665

Open

3 tasks

MarcoPolo approved these changes Feb 28, 2023

View reviewed changes

validation_builtin.go Show resolved Hide resolved

the nonce belongs to the origin peer

ba9e8ba

MarcoPolo approved these changes Feb 28, 2023

View reviewed changes

vyzo added 3 commits February 28, 2023 23:24

add note about rust predicament

af54ade

add seqno validator tests

ee6d237

minor test tweak, ensure at least 1ms before replay

7744fc8

vyzo merged commit 829f902 into master Mar 1, 2023

vyzo deleted the feat/default-validator branch March 1, 2023 14:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Default validator support #525

Default validator support #525

vyzo commented Feb 28, 2023 •

edited

Loading

MarcoPolo left a comment

stbrody commented Mar 1, 2023

vyzo commented Mar 2, 2023 •

edited

Loading

vyzo commented Mar 4, 2023 •

edited

Loading

stbrody commented Mar 5, 2023

vyzo commented Mar 5, 2023 •

edited

Loading

Default validator support #525

Default validator support #525

Conversation

vyzo commented Feb 28, 2023 • edited Loading

MarcoPolo left a comment

Choose a reason for hiding this comment

stbrody commented Mar 1, 2023

vyzo commented Mar 2, 2023 • edited Loading

vyzo commented Mar 4, 2023 • edited Loading

stbrody commented Mar 5, 2023

vyzo commented Mar 5, 2023 • edited Loading

vyzo commented Feb 28, 2023 •

edited

Loading

vyzo commented Mar 2, 2023 •

edited

Loading

vyzo commented Mar 4, 2023 •

edited

Loading

vyzo commented Mar 5, 2023 •

edited

Loading