Does zstd supports sparse files?

[oneumyvakin]

Hello, could you please share does zstd compressor/decompressor aware about sparse files? https://en.wikipedia.org/wiki/Sparse_file
As far as I see here https://manpages.debian.org/unstable/zstd/zstd.1.en.html this implementation is aware about sparse support

--[no-]sparse: enable / disable sparse FS support, to make files with many zeroes smaller on disk. Creating sparse files may save disk space and speed up decompression by reducing the amount of disk I/O. default: enabled when output is into a file, and disabled when output is stdout. This setting overrides default and can force sparse mode over stdout.

[klauspost]

@oneumyvakin That is purely up to you. This package does not handle disk IO in any way.

[oneumyvakin]

I'm right understood that to support sparse files for decompressing there is need to detect zero/null regions and somehow tell/write it into a output file?

[klauspost]

@oneumyvakin I haven't checked how it is actually implemented, but I don't suspect it is too deep. There isn't a huge gain integrating it into the decoder, since it has no special knowledge of '0's.

A simple WriteCloser wrapper that checks for 0 in, say blocks of 8 bytes, should be fast enough to consume input. But you will have to write it yourself.

[SaveTheRbtz]

As mentioned by Klaus: it it easy to wrap the decoder to check for large runs of 0s. Though I would advise a bigger block sizes, e.g. 4096, since this is usually the minimum FS block size at which it can effectively punch holes.