Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFE] Add mount option to control sparse file creation #73

Open
lukts30 opened this issue Feb 23, 2023 · 0 comments
Open

[RFE] Add mount option to control sparse file creation #73

lukts30 opened this issue Feb 23, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@lukts30
Copy link

lukts30 commented Feb 23, 2023

A lot of programs implicitly create sparse files on Linux and while they will work fine under Linux some builtin Windows tools do not support such files at all.
Notable example where Windows does not allow sparse files are Disk Images (*.iso, *.vhd,.vhdx*). [1] [2]

Adding option to every program that might write sparse file under Linux is impractical instead there should be a ntfs3g mount option that prevents new sparse files or adding new write holes to existing files.

Note: The Kernel NTFS3 driver already does not create sparse files by default and the following command creates a disk image that is usable on Windows (mounting or Hyper-V).

qemu-img convert /nvme_pool/vhd_boot_test_dir/WinDev2301Eval.vhdx -O vhdx -o subformat=dynamic /mnt/WinDev2301Eval.vhdx

Running the same command on an NTFS3g mount creates a highly fragmented and sparse file that is not usable under Windows. Only the write holes cause Windows to reject the file.


Current workaround are copying the sparse file with cp --sparse=never or dd if=sparsefile of=sparsefile conv=notrunc bs=1M.
Running qemu-img convert with a preallocated destination file inflates the file to it's full size rendering subformat=dynamic useless.
The only workaround that does not required writing the file twice that I found is to use nbdcopy.

qemu-img info WinDev2301Eval.vhdx
image: WinDev2301Eval.vhdx  
file format: vhdx  
virtual size: 125 GiB (134217728000 bytes)  
disk size: 41.1 GiB  
cluster_size: 33554432

qemu-img create -f vhdx -o subformat=dynamic /mnt/Data/dst.vhdx 125G
# umount 
ntfsfallocate -l 44G /dev/zd48p1 Data/dst.vhdx -n
# mount
cd /mnt/Data/
nbdcopy --destination-is-zero --progress -- [ qemu-nbd -f vhdx ../WinDev2301Eval.vhdx ] [ qemu-nbd -f vhdx dst.vhdx ]

I also tried running ntfsfallocate on the sparse vhdx created with qemu-img convert but ntfsfallocate fails either with NO SPACE LEFT even though the NTFS partition should have enough space left (120GB/150GB) or it crashes corrupting the file system.

[root@c7523d0db1a7 /]# ntfsfallocate -l 45G -n /dev/zd48p1 Data/dst.vhdx
ntfsfallocate v2022.10.3 (libntfs-3g)
File size error : apparent 48318382080, compressed 85771341824 > allocated 48318382080Segmentation fault (core dumped)

# too small also segfaults 
[root@c7523d0db1a7 /]# ntfsfallocate -l 40G -n /dev/zd48p1 Data/dst.vhdx
ntfsfallocate v2022.10.3 (libntfs-3g)
Segmentation fault (core dumped)

image

@unsound unsound added the enhancement New feature or request label Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants