Refactor timecache implementations #523

vyzo · 2023-02-21T11:57:23Z

Refactors the timechache implementations for a much simpler implementation and good behaviour.

In light of #521 and follow up to #522.
Closes #515

MarcoPolo · 2023-02-21T18:38:12Z

timecache/first_seen_cache_test.go

 		tc.Add(fmt.Sprint(i))
 		time.Sleep(time.Millisecond * 100)
 	}

+	time.Sleep(2 * time.Second)


Would be nice if we could use a mock clock here.

yeah... future pr ;)

timecache/time_cache.go

MarcoPolo · 2023-02-21T19:08:55Z

timecache/time_cache.go

 	Has(string) bool
+	// Done signals that the user is done with this cache, which it may stop background threads
+	// and relinquish resources.
+	Done()


Is .Close() not the preferred convention? .Done() is used in context.Context which returns a channel that's closed when the context is finished. .Done() is also used in waitgroups to signal completion (but not cleanup).

.Close() seems closer to the intent.

That said. This is minor, really just a nit

close returns error... there is none here, and there is no point.

Initially i used Closer actually, but then i realize i am making work for myself with the error ;)

Wondertan · 2023-02-21T19:22:56Z

timecache/first_seen_cache.go


-	_, ok := tc.m[s]
-	if ok {
-		log.Debug("first-seen: got same entry")


Since we removed the log here, we can also remove the global log

i guess... doesnt hurt all that much though.

* feat: expire messages from the cache based on last seen time (libp2p#513) * feat: expire messages from the cache based on last seen time * chore: minor renaming * fix: messages should not be found after expiration * chore: editorial * fix: use new time cache strategy consistently * fix: default to old time cache and add todo for background gc * chore: update to go-libp2p v0.25 (libp2p#517) * Update to go-libp2p v0.25 * Use go 1.19 * chore: update go version and dependencies (libp2p#516) * fix(timecache): remove panic in first seen cache on Add (libp2p#522) * Refactor timecache implementations (libp2p#523) * reimplement timecache for sane and performant behaviour * remove seenMessagesMx, take advantage of new tc api * fix timecache tests * fix typo * store expiry, don't make life difficult * refactor common background sweep procedure for both impls * add godocs to TimeCache * Default validator support (libp2p#525) * add default validator support * add an implementation for basic seqno as nonce validation * missing return * the nonce belongs to the origin peer * add note about rust predicament * add seqno validator tests * minor test tweak, ensure at least 1ms before replay * Fix Memory Leak In New Timecache Implementations (libp2p#528) * fix bug * add for last seen cache * chore: Update .github/workflows/stale.yml [skip ci] * chore: Update .github/workflows/stale.yml [skip ci] * upgrades libp2p version * upgrades libp2p version --------- Co-authored-by: Mohsin Zaidi <2236875+smrz2001@users.noreply.github.com> Co-authored-by: Marco Munizaga <git@marcopolo.io> Co-authored-by: RichΛrd <info@richardramos.me> Co-authored-by: Hlib Kanunnikov <hlibwondertan@gmail.com> Co-authored-by: vyzo <vyzo@hackzen.org> Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: GitHub <noreply@github.com>

* feat: expire messages from the cache based on last seen time (libp2p#513) * feat: expire messages from the cache based on last seen time * chore: minor renaming * fix: messages should not be found after expiration * chore: editorial * fix: use new time cache strategy consistently * fix: default to old time cache and add todo for background gc * chore: update to go-libp2p v0.25 (libp2p#517) * Update to go-libp2p v0.25 * Use go 1.19 * chore: update go version and dependencies (libp2p#516) * fix(timecache): remove panic in first seen cache on Add (libp2p#522) * Refactor timecache implementations (libp2p#523) * reimplement timecache for sane and performant behaviour * remove seenMessagesMx, take advantage of new tc api * fix timecache tests * fix typo * store expiry, don't make life difficult * refactor common background sweep procedure for both impls * add godocs to TimeCache * Default validator support (libp2p#525) * add default validator support * add an implementation for basic seqno as nonce validation * missing return * the nonce belongs to the origin peer * add note about rust predicament * add seqno validator tests * minor test tweak, ensure at least 1ms before replay * Fix Memory Leak In New Timecache Implementations (libp2p#528) * fix bug * add for last seen cache * chore: Update .github/workflows/stale.yml [skip ci] * chore: Update .github/workflows/stale.yml [skip ci] * bump golang.org/x/net from 0.4.0 to 0.7.0 (libp2p#520) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](golang/net@v0.4.0...v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: topicscore params can't be set for dynamically subscribed topic (libp2p#540) * fix: topicscore params can't be set for a topic subscribed after gossipsub is initialized * chore:address review comments * Revert "fix: topicscore params can't be set for dynamically subscribed topic (libp2p#540)" (libp2p#541) This reverts commit aa5fd79. * remove usage of deprecated peerid.Pretty method (libp2p#542) * chore: update go-libp2p to v0.32 (libp2p#548) * chore: Update .github/workflows/stale.yml [skip ci] * make tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Mohsin Zaidi <2236875+smrz2001@users.noreply.github.com> Co-authored-by: Marco Munizaga <git@marcopolo.io> Co-authored-by: RichΛrd <info@richardramos.me> Co-authored-by: Hlib Kanunnikov <hlibwondertan@gmail.com> Co-authored-by: vyzo <vyzo@hackzen.org> Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: GitHub <noreply@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com> Co-authored-by: Sukun <sukunrt@gmail.com>

vyzo added 3 commits February 21, 2023 13:42

reimplement timecache for sane and performant behaviour

5b5d18a

remove seenMessagesMx, take advantage of new tc api

ee4bc7a

fix timecache tests

b511356

vyzo requested review from marten-seemann and MarcoPolo February 21, 2023 11:57

vyzo added 3 commits February 21, 2023 14:22

fix typo

b47a8e0

store expiry, don't make life difficult

e671899

refactor common background sweep procedure for both impls

b5f12e3

MarcoPolo approved these changes Feb 21, 2023

View reviewed changes

add godocs to TimeCache

9db0a0b

MarcoPolo approved these changes Feb 21, 2023

View reviewed changes

vyzo merged commit 56c0e6c into master Feb 21, 2023

vyzo deleted the fix/timecache branch February 21, 2023 19:13

Wondertan reviewed Feb 21, 2023

View reviewed changes

nisdas mentioned this pull request Mar 15, 2023

Fix Memory Leak In New Timecache Implementations #528

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor timecache implementations #523

Refactor timecache implementations #523

vyzo commented Feb 21, 2023 •

edited

Loading

MarcoPolo Feb 21, 2023

vyzo Feb 21, 2023

MarcoPolo Feb 21, 2023

vyzo Feb 21, 2023

vyzo Feb 21, 2023 •

edited

Loading

Wondertan Feb 21, 2023

vyzo Feb 21, 2023

Refactor timecache implementations #523

Refactor timecache implementations #523

Conversation

vyzo commented Feb 21, 2023 • edited Loading

MarcoPolo Feb 21, 2023

Choose a reason for hiding this comment

vyzo Feb 21, 2023

Choose a reason for hiding this comment

MarcoPolo Feb 21, 2023

Choose a reason for hiding this comment

vyzo Feb 21, 2023

Choose a reason for hiding this comment

vyzo Feb 21, 2023 • edited Loading

Choose a reason for hiding this comment

Wondertan Feb 21, 2023

Choose a reason for hiding this comment

vyzo Feb 21, 2023

Choose a reason for hiding this comment

vyzo commented Feb 21, 2023 •

edited

Loading

vyzo Feb 21, 2023 •

edited

Loading