Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Raw encrypted send stream can't be dry-run received: invalid record type #9173

Closed
implr opened this issue Aug 17, 2019 · 0 comments · Fixed by #9221
Closed

Raw encrypted send stream can't be dry-run received: invalid record type #9173

implr opened this issue Aug 17, 2019 · 0 comments · Fixed by #9221
Labels
Component: Encryption "native encryption" feature Component: Send/Recv "zfs send/recv" feature

Comments

@implr
Copy link

implr commented Aug 17, 2019

System information

Type Version/Name
Distribution Name Gentoo
Distribution Version ~amd64
Linux Kernel 5.1.15-gentoo
Architecture amd64
ZFS Version 0.8.1-r0-gentoo
SPL Version 0.8.1-r0-gentoo

Describe the problem you're observing

zfs recv -n fails with invalid record type on streams created by send -w from encrypted datasets. The stream itself is fine and receives correctly when not doing a dry run.

Describe how to reproduce the problem

Create two test pools, an encrypted dataset on the first, snapshot and raw send it:

/tmp # dd if=/dev/zero bs=4k count=50k of=pool1
/tmp # dd if=/dev/zero bs=4k count=50k of=pool2
/tmp # zpool create pool1 /tmp/pool1 
/tmp # zpool create pool2 /tmp/pool2
/tmp # zfs create -o encryption=on -o keyformat=passphrase pool1/crypt
Enter passphrase: 
Re-enter passphrase: 
/tmp # dd if=/dev/urandom of=/pool1/crypt/foo bs=4k count=5k
/tmp # zfs snapshot pool1/crypt@bar
/tmp # zfs send -vw pool1/crypt@bar > send.img
full send of pool1/crypt@bar estimated size is 20.1M
total estimated size is 20.1M
TIME        SENT   SNAPSHOT pool1/crypt@bar

Try to dry-run receive it in the second pool:

/tmp # zfs recv -ne pool2 < send.img
cannot receive:: invalid record type

Actually receiving it works fine:

/tmp # zfs recv -e pool2 < send.img 
/tmp # zfs load-key pool2/crypt
Enter passphrase for 'pool2/crypt': 
/tmp # zfs mount pool2/crypt       
/tmp # diff -rs /pool1/crypt/ /pool2/crypt/
Files /pool1/crypt/foo and /pool2/crypt/foo are identical

Include any warning/errors/backtraces from the system logs

Nothing in dmesg.
@behlendorf behlendorf added Component: Encryption "native encryption" feature Component: Send/Recv "zfs send/recv" feature labels Aug 27, 2019
tcaputi pushed a commit to datto/zfs that referenced this issue Aug 27, 2019
Fix noop receive of raw send stream
277b806 
Currently, the noop receive code fails to work with raw send streams
and resuming send streams. This happens because zfs_receive_impl()
reads the DRR_BEGIN payload without reading the payload itself.
Normally, the kernel expects to read this itself, but in this case
the recv_skip() code runs instead and it is not prepared to handle
the stream being left at any place other than the beginning of a
record.

This patch resolves this issue by manually reading the DRR_BEGIN
payload in the dry-run case. This patch also includes a number of
small fixups in this code path.

Fixes: openzfs#9173

Signed-off-by: Tom Caputi <tcaputi@datto.com>
@tcaputi tcaputi mentioned this issue Aug 27, 2019
Merged
12 tasks
tcaputi pushed a commit to datto/zfs that referenced this issue Aug 28, 2019
Fix noop receive of raw send stream
fa0c53d 
Currently, the noop receive code fails to work with raw send streams
and resuming send streams. This happens because zfs_receive_impl()
reads the DRR_BEGIN payload without reading the payload itself.
Normally, the kernel expects to read this itself, but in this case
the recv_skip() code runs instead and it is not prepared to handle
the stream being left at any place other than the beginning of a
record.

This patch resolves this issue by manually reading the DRR_BEGIN
payload in the dry-run case. This patch also includes a number of
small fixups in this code path.

Fixes: openzfs#9173

Signed-off-by: Tom Caputi <tcaputi@datto.com>
tcaputi pushed a commit to datto/zfs that referenced this issue Sep 5, 2019
Fix noop receive of raw send stream
4d5f236 
Currently, the noop receive code fails to work with raw send streams
and resuming send streams. This happens because zfs_receive_impl()
reads the DRR_BEGIN payload without reading the payload itself.
Normally, the kernel expects to read this itself, but in this case
the recv_skip() code runs instead and it is not prepared to handle
the stream being left at any place other than the beginning of a
record.

This patch resolves this issue by manually reading the DRR_BEGIN
payload in the dry-run case. This patch also includes a number of
small fixups in this code path.

Fixes: openzfs#9173

Signed-off-by: Tom Caputi <tcaputi@datto.com>
behlendorf pushed a commit that referenced this issue Sep 5, 2019
@behlendorf
Fix noop receive of raw send stream
870e7a5 
Currently, the noop receive code fails to work with raw send streams
and resuming send streams. This happens because zfs_receive_impl()
reads the DRR_BEGIN payload without reading the payload itself.
Normally, the kernel expects to read this itself, but in this case
the recv_skip() code runs instead and it is not prepared to handle
the stream being left at any place other than the beginning of a
record.

This patch resolves this issue by manually reading the DRR_BEGIN
payload in the dry-run case. This patch also includes a number of
small fixups in this code path.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #9221
Closes #9173
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Dec 24, 2019
@tonyhutter
Fix noop receive of raw send stream
5a41c8e 
Currently, the noop receive code fails to work with raw send streams
and resuming send streams. This happens because zfs_receive_impl()
reads the DRR_BEGIN payload without reading the payload itself.
Normally, the kernel expects to read this itself, but in this case
the recv_skip() code runs instead and it is not prepared to handle
the stream being left at any place other than the beginning of a
record.

This patch resolves this issue by manually reading the DRR_BEGIN
payload in the dry-run case. This patch also includes a number of
small fixups in this code path.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes openzfs#9221
Closes openzfs#9173
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Dec 27, 2019
@tonyhutter
Fix noop receive of raw send stream
186abea 
Currently, the noop receive code fails to work with raw send streams
and resuming send streams. This happens because zfs_receive_impl()
reads the DRR_BEGIN payload without reading the payload itself.
Normally, the kernel expects to read this itself, but in this case
the recv_skip() code runs instead and it is not prepared to handle
the stream being left at any place other than the beginning of a
record.

This patch resolves this issue by manually reading the DRR_BEGIN
payload in the dry-run case. This patch also includes a number of
small fixups in this code path.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes openzfs#9221
Closes openzfs#9173
tonyhutter pushed a commit that referenced this issue Jan 23, 2020
@tonyhutter
Fix noop receive of raw send stream
ffe29e7 
Currently, the noop receive code fails to work with raw send streams
and resuming send streams. This happens because zfs_receive_impl()
reads the DRR_BEGIN payload without reading the payload itself.
Normally, the kernel expects to read this itself, but in this case
the recv_skip() code runs instead and it is not prepared to handle
the stream being left at any place other than the beginning of a
record.

This patch resolves this issue by manually reading the DRR_BEGIN
payload in the dry-run case. This patch also includes a number of
small fixups in this code path.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #9221
Closes #9173
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component: Encryption "native encryption" feature Component: Send/Recv "zfs send/recv" feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix noop receive of raw send stream datto/zfs
2 participants
@behlendorf @implr