Memory usage (sometimes 15x from the size of dicom file)

[vitalii-komenda]

dicom.ParseFile() method expands memory a lot. I have a service that gets OOM killed because of this method uses so much memory

[suyashkumar]

Thanks for the info! Any more details you can share on your use case? Note that we have options to not read (or not parse) PixelData if that makes sense for your use case. #267 might also help, depending on what you're trying to do e.g if you're doing element by element processing you will never have to hold the whole dicom in memory at once. Also, what version of the library are you using?

See more performance discussion in #161 (in particular #161 (comment)), but there's a known situation where underlying native dicom integer data will be expanded from smaller ints (e.g. int8s) to int (usually int64, mostly in the name of having a consistent API for pixel data), but we might be able to hold the data as a Wrapper[T] where T is {int8, int16, int32, etc} at some point in the future. Also note the data is copied when generating an image.Image (here) for now (which for now takes only a uint16).

I'll take another pass at ways to reduce memory footprint for native images without compromising the API too much, esp now that we have generics.

If you're using the native pixel data, I'd love to know a little more about the use case as well! Are you working with the raw values at all?

[suyashkumar]

Some updates: in #315 (comment) you'll see an implementation that yields significant memory and cpu improvements for native pixeldata processing. I still need to iterate and clean up some tests / api though. Please let me know if you have any thoughts in general here though. This will be a pretty significant change to how Native PixelData is represented and interacted with.