You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sophisticated analyses use high-dimensional histograms, both disc and memory usage can become an issue. Disc usage is easy to fix by writing a compressed stream with Boost.Serialization and Boost.IOStreams. But this does not fix the memory issue.
It would be interesting to add an experimental memory-compressed storage and see how that performs. In-memory compression can be achieved in several ways.
Zero-suppression: Some high-dimensional histograms contain a lot of zeros. A simple compression would just replace all the zeros with a code that indicates how many zeros are in the omitted gap. This is comparably easy to implement. Frequent re-allocations of memory in the beginning will be an issue, but that could be optimized over time.
Sophisticated analyses use high-dimensional histograms, both disc and memory usage can become an issue. Disc usage is easy to fix by writing a compressed stream with Boost.Serialization and Boost.IOStreams. But this does not fix the memory issue.
It would be interesting to add an experimental memory-compressed storage and see how that performs. In-memory compression can be achieved in several ways.
The text was updated successfully, but these errors were encountered: