-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: shared source inside chunked_by adapters #4232
Conversation
PRO-925 Use shared adapter inside the chunking adapters
In order to not duplicate the work of the upstream adapters after chunking, the chain source that's passed into the chunking adapter needs to be shared. Rather than have to remember this needs to be done, we should instead do the sharing inside the chunking adapters themselves so it can't be forgotten. The reason for this is that the chunked adapters create a separate client and stream for each epoch, the shared adapter ensures they are all using the same underlying stream and client. |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #4232 +/- ##
======================================
- Coverage 72% 71% -0%
======================================
Files 384 384
Lines 63015 63185 +170
Branches 63015 63185 +170
======================================
+ Hits 45074 45123 +49
- Misses 15609 15728 +119
- Partials 2332 2334 +2 ☔ View full report in Codecov by Sentry. |
I agree, think it's fine if it's used somewhere it's not necessary for the benefit of never forgetting to include it (which is a worse consequence) |
* refactor: shared source inside chunked_by adapters * chore: address review
Pull Request
Closes: PRO-925
Checklist
Please conduct a thorough self-review before opening the PR.
Summary
One downside of this approach is that
shared
is sometimes called more times than strictly necessary (when shared source was used by more than just the chunked_by adapter, e.g. onstrictly_monotonic_source
in btc witnesser). The alternative I considered is to make chunked_by adapters methods on SharedSource directly, but I think current solution is OK considering that shared adapter seems relatively lightweight, and moving calls toshared
cleaned up the high level code a little.