Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memoize Keccaks #7333

Closed
wants to merge 6 commits into from
Closed

Memoize Keccaks #7333

wants to merge 6 commits into from

Conversation

benaadams
Copy link
Member

@benaadams benaadams commented Aug 15, 2024

Follow up to #7328

Changes

  • Memoize Keccaks (.Length <= 32)
    • 16 way partitioned ClockCache (LRU); partitioned to reduce contention (adds are frequent)
    • Though very random same Keccak can be used multiple times, whether from
      • Parallel tx exectuions (same code run twice)
      • Multiple use of same contract keccak(address)
      • Same slot across many contracts in solidity mapping i.e. keccak(slot)
  • Reduce ClockCache contention
    • Calling ConcurrentDictionary.Count takes locks and causes contention on read side
    • Use local count that is only changed when have write lock anyway
  • More variance in Bytes GetHashCode

Types of changes

What types of changes does your code introduce?

  • Optimization

Testing

Requires testing

  • No

src/Nethermind/Nethermind.Core/Extensions/Bytes.cs Outdated Show resolved Hide resolved
src/Nethermind/Nethermind.Core/Extensions/Bytes.cs Outdated Show resolved Hide resolved
src/Nethermind/Nethermind.Core/Crypto/Keccak.cs Outdated Show resolved Hide resolved
src/Nethermind/Nethermind.Core/Crypto/Keccak.cs Outdated Show resolved Hide resolved
{
internal readonly byte[] _bytes = bytes;
public static implicit operator BytesWrapper(byte[] bytes) => new(bytes);
public static implicit operator BytesWrapper(ReadOnlySpan<byte> bytes) => new(bytes.ToArray());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we search by Span without allocating first, like in SpanDictionary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in the regular ConcurrentDictionary; though it should live an die pretty fast in Gen0.

Also another reason to limiting to 32 bytes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well it is good I made SpanConcurrentDictionary ]:->

Copy link
Member

@LukaszRozmej LukaszRozmej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be great to use SpanDictionary in the cache (maybe have a derived type? And avoid allocating when searching in cache.

@kamilchodola
Copy link
Contributor

Will throw that for comaprision with master on same spec

Copy link
Contributor

@Scooletz Scooletz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor remarks while working on a different approach.

{
private const int CacheCount = 256;
private const int CacheMax = CacheCount - 1;
private readonly ClockCache<BytesWrapper, ValueHash256>[] _caches;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite a lot of objects here 😢

@benaadams
Copy link
Member Author

benaadams commented Aug 19, 2024

Closing in favour of #7336

@benaadams benaadams closed this Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants