You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Once #231 is complete, SpanManager needs to stop having the compressed spans cache and keep caching only uncompressed spans going forward. Compressed spans should be cached within BlobReader. Please refer to the design doc from #103 for more details on the concepts.
In short, we need to do the following:
BlobReader now has the cache of spans (aka regions).
SpanManager keeps only the cache of uncompressed spans. If it needs the compressed span, it requests it from BlobReader.
Describe the solution you'd like
The outcome for this issue is a design for the separation.
Describe alternatives you've considered
Keep everything as is.
Kern--
changed the title
Refactor the data path for soci snapshotter: part 2
Move caching of compressed data from SpanManager to BlobReader
Dec 13, 2022
Kern--
changed the title
Move caching of compressed data from SpanManager to BlobReader
Migrate caching of compressed data from SpanManager to BlobReader
Dec 13, 2022
While we're doing this work, we should be thinking about the ability to decouple the actual network requests from the span size. A span is really a unit of decompression, but we also use it as a unit of data to be fetched from the network. We can probably get some better performance if we optimize our network requests separately from the decompression. E.g. S3 likes 8 or 16MiB requests, but we might want 2 or 4MiB spans to reduce necessary compute.
We don't actually have to solve this problem in the initial separation, but we shouldn't design it in such a way that we can't do that later.
Is your feature request related to a problem? Please describe.
Once #231 is complete, SpanManager needs to stop having the compressed spans cache and keep caching only uncompressed spans going forward. Compressed spans should be cached within BlobReader. Please refer to the design doc from #103 for more details on the concepts.
In short, we need to do the following:
Describe the solution you'd like
The outcome for this issue is a design for the separation.
Describe alternatives you've considered
Keep everything as is.
Additional context
#103
The text was updated successfully, but these errors were encountered: