-
Notifications
You must be signed in to change notification settings - Fork 924
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pruner: Implement skeleton for pruner package #2972
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i preferred when you named this a carcass, but skeleton makes sense :-)
Really my only genuine comments are that i'm stuck on naming, i wonder if the package should be called storage or something? Thinking out loud on naming, but maybe it's:
- storage (ie: concerns related to storage of blocks)
- storage/pruner = where pruner implements the various 'workers' implementations that performs pruning for diff node types
- storage/availability (which then implements a range of Window functions or Sampling strategies for node types)
Thinking then it conceptually reads like
Daser, is injected with an availabilityWindowSampler, which checks sample/storage conditions based on the header coming in, and then storage has pruner package "pruning" which implements various strategies for pruning data which is adjacent to windowSampler.
Thinking this is not dissimilar to OTEL and trace sampling etc
is there a better name than storage as top level if we went with that?
so then pruning.AvailabilityWindow
-> availability.WindowSampler
, and maybe has sampler.ShouldSample(h Header)
true etc?
all suggestions. also, reading about pruning so much makes me think of baby food.
be09edc
to
fea6c4a
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #2972 +/- ##
==========================================
+ Coverage 50.92% 50.94% +0.01%
==========================================
Files 168 168
Lines 10944 11038 +94
==========================================
+ Hits 5573 5623 +50
- Misses 4872 4913 +41
- Partials 499 502 +3 ☔ View full report in Codecov by Sentry. |
…s handed to daser for sampling window
I've removed How do we feel about this approach? Benefits: it does not break config for DASer and it only would introduce a config where actually necessary/allowable (full-pruning). |
Another benefit - enabling sampling window for light nodes will be as simple as changing this one line: https://github.com/celestiaorg/celestia-node/pull/2972/files#diff-67d0619a422b8d0b0254a8f10165ae1d47bcc142b7fbd3e61591262f448731baR42 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kinda love this tbh
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great 🚀
This PR implements a skeleton for the pruner package and integrates the pruner's AvailabilityWindow determination into the DASer sampling routine.
Namings are up for discussion.