You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a (very big) performance issue when saving an OME TIFF file with PyramidOMETiffWriter as the close() method duration scales as the number of planes squared.
I've tested to save a 512x512 pixel file on my computer and increase the number of timepoints by powers of 2 (nT). I've measured the time required to save the file and split the required time between what happens before calling close, and the close duration:
Export Time (ms) = Data writing (ms) + Reader.close() duration (ms)
nPixX
nPixY
nT
Export Time (s)
Reader.close() duration (s)
Data writing (s)
Reader close duration in total export time (%)
512
512
1
0.5
0.2
0.4
32 %
512
512
2
0.7
0.2
0.5
27 %
512
512
4
0.7
0.2
0.5
32 %
512
512
8
1.0
0.3
0.7
30 %
512
512
16
1.4
0.4
1.0
29 %
512
512
32
2.1
0.7
1.4
31 %
512
512
64
3.8
1.7
2.2
43 %
512
512
128
8.9
5.2
3.7
59 %
512
512
256
20.2
14.4
5.8
71 %
512
512
512
64.0
51.6
12.5
80 %
512
512
1024
290.4
262.6
27.8
90 %
16384
16384
1
86.2
0.08
86.1
0.1 %
The data writing time scales linearly with the number of timepoints, which is what we expect. However the close time is multiplied by 4 each time the number of timepoints is multiplied by 2.
As a result, 'close' takes 90% of the total export time when there are 1024 timepoints.
I've narrowed the issue to the call of TiffParser#getIFDOffsets for all planes, getIFDOffsets walking the whole file from the start. This scales very badly.
PyramidOMETiffWriter#close loops through all planes:
Hello,
Original post: https://forum.image.sc/t/pyramidometiffwriter-really-bad-performance-when-closing-the-writer/95294
There is a (very big) performance issue when saving an OME TIFF file with
PyramidOMETiffWriter
as theclose()
method duration scales as the number of planes squared.I've tested to save a 512x512 pixel file on my computer and increase the number of timepoints by powers of 2 (nT). I've measured the time required to save the file and split the required time between what happens before calling close, and the close duration:
The data writing time scales linearly with the number of timepoints, which is what we expect. However the close time is multiplied by 4 each time the number of timepoints is multiplied by 2.
As a result, 'close' takes 90% of the total export time when there are 1024 timepoints.
I've narrowed the issue to the call of
TiffParser#getIFDOffsets
for all planes,getIFDOffsets
walking the whole file from the start. This scales very badly.PyramidOMETiffWriter#close
loops through all planes:bioformats/components/formats-bsd/src/loci/formats/out/PyramidOMETiffWriter.java
Line 125 in 7ca9980
TiffSaver#overwriteIFDValue
bioformats/components/formats-bsd/src/loci/formats/out/PyramidOMETiffWriter.java
Line 145 in 7ca9980
TiffParser#getIFDOffsets
bioformats/components/formats-bsd/src/loci/formats/tiff/TiffSaver.java
Line 804 in 7ca9980
TiffParser#getIFDOffsets
walks through the whole filebioformats/components/formats-bsd/src/loci/formats/tiff/TiffParser.java
Lines 362 to 363 in 7ca9980
Is there a way to fix this ?
This issue could be related to these other issues:
#3983
#3480
The text was updated successfully, but these errors were encountered: