Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(swarm): improve PeerAddresses configurability #5574

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

stormshield-frb
Copy link
Contributor

Description

Our average peer has over 10 advertised addresses. Doing so, identify was continuously (every 1 minute) emitting ToSwarm::NewExternalAddrOfPeer which was overflowing our logs and was making the PeerCache kind of useless.

For this reason, we made some changes in order to be able to configure the number of addresses that should be cached for a peer. We are upstreaming this changes.

Notes & open questions

N/A

Change checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • A changelog entry has been made in the appropriate crates

@stormshield-frb stormshield-frb force-pushed the feat/identify-make-peer-cache-configurable branch 3 times, most recently from 8fbd19b to be31d42 Compare August 28, 2024 15:22
@stormshield-frb stormshield-frb force-pushed the feat/identify-make-peer-cache-configurable branch from be31d42 to e5435fc Compare August 28, 2024 15:36
Copy link
Member

@dariusc93 dariusc93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Left a couple of comments

/// Configures the size of the LRU cache, caching addresses of discovered peers.
pub fn with_cache_size(mut self, cache_size: usize) -> Self {
self.cache_size = cache_size;
/// Configuration for the LRU cache responsible for caching addresses of discovered peers.
///
/// If set to [`None`], caching is disabled.
pub fn with_cache_config(mut self, cache_config: Option<PeerAddressesConfig>) -> Self {
self.cache_config = cache_config;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably keep with_cache_size, deprecating it for with_cache_config and just have use that internally so we could probably introduce this as a patch update. Thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CC @jxs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is possible, I'm definitely doing that. I did not think of that thanks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's done. I'll let you resolve this if you're ok with it. 😉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. It was slightly more complicated than expected because since every attributes of identify::Config are public, I cannot remove cache_size if we want to stay in 0.45.1 and not upgrade to 0.46.0.

Just a thought, but I checked the other "Config" object across the project and some do expose the attributes but most of them don't. Maybe we should homogenize that and make every attributes private to not have this kind of problem later on ? It would be a pain to make many new major updates but it would once and for all. If you think there is something here, I will happily open an issue to discuss this mater more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dariusc93 I'm closing this conversation. Feel free to re-open it if you have other suggestions about it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with François on this one, can we just introduce a breaking change, and take the minor release cycle to also do #5660?

swarm/src/behaviour/peer_addresses.rs Show resolved Hide resolved
Copy link
Contributor

mergify bot commented Sep 5, 2024

This pull request has merge conflicts. Could you please resolve them @stormshield-frb? 🙏

@stormshield-frb stormshield-frb force-pushed the feat/identify-make-peer-cache-configurable branch from 14c1974 to 4d17c21 Compare September 6, 2024 07:57
@stormshield-frb stormshield-frb force-pushed the feat/identify-make-peer-cache-configurable branch from 1b14122 to 457c063 Compare September 6, 2024 08:04
Copy link
Contributor

mergify bot commented Sep 26, 2024

This pull request has merge conflicts. Could you please resolve them @stormshield-frb? 🙏

Copy link
Member

@jxs jxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi François, thanks for this! And sorry for the delay, left some comments

}

/// Configure the maximum number of cached addresses per peer.
pub fn with_number_of_addresses_by_peer(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub fn with_number_of_addresses_by_peer(
pub fn with_number_of_addresses_per_peer(

Copy link
Contributor

mergify bot commented Nov 6, 2024

This pull request has merge conflicts. Could you please resolve them @stormshield-frb? 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants