An interrupted initial raw zfs send can not be resumed with reported token: error 1030 IV set guid missing

[IslandLife]

System information

Type	Version/Name
Distribution Name	Ubuntu
Distribution Version	19.10 eoan
Linux Kernel	5.3.0-46-generic
Architecture	amd64
ZFS Version	0.8.0-731_g1f043c8be
SPL Version	0.8.0-731_g1f043c8be

Describe the problem you're observing

An interrupted initial full raw zfs send can not be resumed with reported token:
If you attempt to resume using the provided token you get an error message:

cannot receive resume stream: IV set guid missing. See errata 4 at https://zfsonlinux.org/msg/ZFS-8000-ER.

This behaviour is seen on a raw send but not on a simple send. A simple send can be resumed OK.

Describe how to reproduce the problem

1. In a piped command use dd between send and recv to send only a finite number of blocks of the stream. The send/recv will of course fail. Record the reported token from the recv process.
2. Attempt to resume using the provided token from the previous step.
   Example:

root@slosanhd4:/etc/opt/bin# zfs send -w -v  brown/vol-drbd1@20200421.0952-1d | dd count=1000000 | zfs recv -s  brown/dummy
full send of brown/vol-drbd1@20200421.0952-1d estimated size is 811G
total estimated size is 811G
TIME        SENT   SNAPSHOT brown/vol-drbd1@20200421.0952-1d
10:02:45   1.07M   brown/vol-drbd1@20200421.0952-1d
10:02:46    141M   brown/vol-drbd1@20200421.0952-1d
10:02:47    353M   brown/vol-drbd1@20200421.0952-1d
10:02:48    485M   brown/vol-drbd1@20200421.0952-1d
999818+182 records in
999914+1 records out
511956132 bytes (512 MB, 488 MiB) copied, 4.87804 s, 105 MB/s
cannot receive new filesystem stream: checksum mismatch or incomplete stream.
Partially received snapshot is saved.
A resuming stream can be generated on the sending system by running:
    zfs send -t 1-11c16bddc1-108-789c636064000310a500c4ec50360710e72765a5269740f80cd8e4d3d28a534b4032b3fff340e5d990e4932a4b528b8174c8a466396cfa4bf2d34b33531818162b6e987dd83b33d901499e132c9f97989b0a34a728bf3c4fbf2c3f4737a52829c5d0c1c8c0c8c0c0c4c850cfc0d2d448d7308501c97ddc0c08ff24e7e71614a51617e76743c424a0ee83c9172596c3a41800ebaa2441

The provided token appears valid, send -nv does interpret it

root@slosanhd4:/etc/opt/bin# zfs send -nv -t 1-11c16bddc1-108-789c636064000310a500c4ec50360710e72765a5269740f80cd8e4d3d28a534b4032b3fff340e5d990e4932a4b528b8174c8a466396cfa4bf2d34b33531818162b6e987dd83b33d901499e132c9f97989b0a34a728bf3c4fbf2c3f4737a52829c5d0c1c8c0c8c0c0c4c850cfc0d2d448d7308501c97ddc0c08ff24e7e71614a51617e76743c424a0ee83c9172596c3a41800ebaa2441
resume token contents:
nvlist version: 0
	object = 0x1
	offset = 0xcff9b0000
	bytes = 0x1e839254
	toguid = 0x63694bc39bb021a3
	toname = brown/vol-drbd1@20200421.0952-1d
	compressok = 1
	rawok = 1
cannot resume send: Unknown error 1030

However it can not be used

root@slosanhd4:/etc/opt/bin# zfs send -v -t 1-11c16bddc1-108-789c636064000310a500c4ec50360710e72765a5269740f80cd8e4d3d28a534b4032b3fff340e5d990e4932a4b528b8174c8a466396cfa4bf2d34b33531818162b6e987dd83b33d901499e132c9f97989b0a34a728bf3c4fbf2c3f4737a52829c5d0c1c8c0c8c0c0c4c850cfc0d2d448d7308501c97ddc0c08ff24e7e71614a51617e76743c424a0ee83c9172596c3a41800ebaa2441 | dd count=1000000 | zfs recv -s  brown/dummy
resume token contents:
nvlist version: 0
	object = 0x1
	offset = 0xcff9b0000
	bytes = 0x1e839254
	toguid = 0x63694bc39bb021a3
	toname = brown/vol-drbd1@20200421.0952-1d
	compressok = 1
	rawok = 1
cannot resume send: Unknown error 1030
cannot receive resume stream: IV set guid missing. See errata 4 at https://zfsonlinux.org/msg/ZFS-8000-ER.

The use of dd and testing if the reported token can be used and possibly a whole series of interrupted and resumed streams is something which could be used in a test script.

[ahrens]

cc @tcaputi

[tcaputi]

@IslandLife assuming youre ok building your own zfs would you mind giving this PR a spin? #10239

[IslandLife]

I applied your patch and it works now. Thanks!