Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement NFSv41 ACLs through xattr #9

Closed
wants to merge 27 commits into from
Closed

Implement NFSv41 ACLs through xattr #9

wants to merge 27 commits into from

Conversation

anodos325
Copy link

This implements NFSv41 (RFC 5661) ACLs in a manner
compatible with vfs_nfs4acl_xattr in Samba and
nfs4xdr-acl-tools.

There are three key areas of change in this commit:

  1. NFSv4 ACL management through system.nfs4_acl_xdr xattr.
    Install an xattr handler for "system.nfs4_acl_xdr" that
    presents an xattr containing full NFSv41 ACL structures
    generated through rpcgen using specification from the Samba
    project. This xattr is used by userspace programs to read and
    set permissions.

  2. add an i_op->permissions endpoint: zpl_permissions(). This
    is used by the VFS in Linux to determine whether to allow /
    deny an operation. Wherever possible, we try to avoid having
    to call zfs_access(). If kernel has NFSv4 patch for VFS, then
    perform more complete check of avaiable access mask.

  3. add capability-based overrides to secpolicy_vnode_access2()
    there are various situations in which ACL may need to be
    overridden based on capabilities. This logic is almost directly
    copied from Linux VFS. For instance, root needs to be able to
    always read / write ACLs (otherwise admin can get locked out
    from files).

This is commit was initially inspired by work from Paul B. Henson
to implement NFSv4.0 (RFC3530) ACLs in ZFS on Linux. Key areas of
divergence are as follows:

  • ACL specification, xattr format, xattr name
  • Addition of handling for NFSv4 masks from Linux VFS
  • Addition of ACL overrides based on capabilities

Signed-off-by: Andrew Walker awalker@ixsystems.com

mattmacy and others added 24 commits April 5, 2021 16:59
@mattmacy
Fix ZFS_DEBUG_MODIFY assert in arc_buf_try_copy_decompressed_data
4ce12a4 
The assert does not account for the case where there is a single
buffer in the chain that is decompressed and has a valid
checksum.

Signed-off-by: Matt Macy <mmacy@FreeBSD.org>
Use setproctitle to report progress of zfs send
c05bed2 
This allows parsing of zfs send progress by checking the process
title.

Doing so requires some changes to the send code in libzfs_sendrecv.c;
primarily these changes move some of the accounting around, to allow
for the code to be verbose as normal, or set the process title. Unlike
BSD, setproctitle() isn't standard in Linux; I found a reference to it
in libbsd, and included autoconf-related changes to test for that.

Authored-by: Sean Eric Fagan <sef@FreeBSD.org>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Add zfsd for FreeBSD
60add51 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Add rc.d/zfs script
7c19e15
Add packaging bits for TrueNAS SCALE
6385c0d
@sonicaj
Add CI for building zfs package
2339b56 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Update changelog for TrueNAS SCALE deb 2.0.1
0460d34 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Update changelog for TrueNAS SCALE deb 2.0.2
2ab2387 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
@kmoore134
Bump release to -3
46fd796 
Fixes an issue with zfs-utils being unable to install due to a mismatch
with upstream Debian.
Update changelog for TrueNAS SCALE deb 2.0.3
f5180f8 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
@sonicaj
Add dockerfile for zfs
1b0612d 
Signed-off-by: Waqar Ahmed <waqarahmedjoyia@live.com>
@sonicaj
Add action to build and push docker image on master update
d18b4c9 
Signed-off-by: Waqar Ahmed <waqarahmedjoyia@live.com>
@sonicaj
Use stable tag until upstream alien package is fixed
ce857d2 
Signed-off-by: Waqar Ahmed <waqarahmedjoyia@live.com>
@sonicaj
Update branch name for docker image CI
1fac720
@anodos325
Add check for custom TrueNAS kernel
64d7701 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Update changelog for TrueNAS SCALE deb 2.0.4
d13fcb4 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
@anodos325
Advertise support for large xattrs on TrueNAS
114d18c 
SB_LARGEXATTR is used in TrueNAS SCALE to indicate to the kernel
that the filesystem supports large-size xattrs (greater than 64KiB).

This flag is used to evaluate whether to allow large xattr read
or write requests (up to 2 MiB).

Signed-off-by: Andrew Walker <awalker@ixsystems.com>
@gmelikov
CI checkstyle: pin ubuntu version
54a4da2 
Our checkstyle doesn't work well on Ubuntu 20.04,
temporary pin it to 18.04.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes openzfs#11713
ZTS: Use ksh and current environment for user_run
985187b 
The current user_run often does not work as expected.  Commands are run
in a different shell, with a different environment, and all output is
discarded.

Simplify user_run to retain the current environment, eliminate eval,
and feed the command string into ksh.  Enhance the logging for
user_run so we can see out and err.

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes openzfs#11185
ZTS: Fix incorrect use of libtest in user_run by xattr_003_neg
417fa4e 
You can't use user_run to eval ksh functions defined in libtest unless
you include libtest in the user shell.

Fix xattr_003_neg by:
* include libtest in the user shell
* *then* run get_xattr
* assert this fails
* use variables for filenames so they don't change in the user's shell
* don't log the contents of /etc/passwd
* cleanup all byproducts

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes openzfs#11185
ZTS: Add tests for DOS mode attributes
e1ed850 
Create a new section of tests to run with acltype=off.

For now the only test we have is for the DOS mode READONLY attribute on
FreeBSD.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes openzfs#11734
@anodos325
Fix regression in POSIX mode behavior
2af305a 
Commit 235a856 introduced a regression in evaluation of POSIX modes
that require group DENY entries in the internal ZFS ACL. An example
of such a POSX mode is 007. When write_implies_delete_child is set,
then ACE_WRITE_DATA is added to `wanted_dirperms` in prior to calling
zfs_zaccess_common(). This occurs is zfs_zaccess_delete().

Unfortunately, when zfs_zaccess_aces_check hits this particular DENY
ACE, zfs_groupmember() is checked to determine whether access should be
denied, and since zfs_groupmember() always returns B_TRUE on Linux and
so this check is failed, resulting ultimately in EPERM being returned.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Andrew Walker <awalker@ixsystems.com>
Closes openzfs#11760
Work around broken alien version
79bb891 
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
@anodos325
Implement NFSv41 ACLs through xattr
85dcd0c 
This implements NFSv41 (RFC 5661) ACLs in a manner
compatible with vfs_nfs4acl_xattr in Samba and
nfs4xdr-acl-tools.

There are three key areas of change in this commit:
1) NFSv4 ACL management through system.nfs4_acl_xdr xattr.
  Install an xattr handler for "system.nfs4_acl_xdr" that
  presents an xattr containing full NFSv41 ACL structures
  generated through rpcgen using specification from the Samba
  project. This xattr is used by userspace programs to read and
  set permissions.

2) add an i_op->permissions endpoint: zpl_permissions(). This
  is used by the VFS in Linux to determine whether to allow /
  deny an operation. Wherever possible, we try to avoid having
  to call zfs_access(). If kernel has NFSv4 patch for VFS, then
  perform more complete check of avaiable access mask.

3) add capability-based overrides to secpolicy_vnode_access2()
  there are various situations in which ACL may need to be
  overridden based on capabilities. This logic is almost directly
  copied from Linux VFS. For instance, root needs to be able to
  always read / write ACLs (otherwise admin can get locked out
  from files).

This is commit was initially inspired by work from Paul B. Henson
to implement NFSv4.0 (RFC3530) ACLs in ZFS on Linux. Key areas of
divergence are as follows:
- ACL specification, xattr format, xattr name
- Addition of handling for NFSv4 masks from Linux VFS
- Addition of ACL overrides based on capabilities

Signed-off-by: Andrew Walker <awalker@ixsystems.com>
@anodos325 anodos325 added the WIP Work In Progress label Apr 28, 2021
@anodos325 anodos325 requested review from a user and rick-mesta April 28, 2021 18:45
@anodos325
Copy link
Author

This is currently WIP branch. Flagging for review because all current regression tests are passing.

@anodos325
Ensure ACE_IDENTIFIER_GROUP is set when ACE_GROUP is set
6525bd0 
In some situations samba's nfs4acl_xattr module will set ACE_GROUP
without setting ACE_IDENTIFIER_GROUP. This will cause zfs_setacl()
to fail with EINVAL. Since the two are always set together, it's
simpler to just always ensure ACE_IDENTIFIER_GROUP is set when
ACE_GROUP is.
@anodos325
Add NULL DACL support
aa23646 
Some applications over the SMB protocol will get / set ACLs with
no entries in them. Rather than fail with EINVAL, convert the
ACL to contain single owner@ entry with an empty mask and flags.
This avoids having to touch ZFS ACL logic (only affects getting /
setting thorugh xattr). If we have a single ACE and it's this one,
strip it off of the returned xattr.
@anodos325 anodos325 force-pushed the NAS-103664 branch 3 times, most recently from 81fd761 to 3c3976b Compare May 2, 2021 02:53
@anodos325
Revert change to add NULL DACL handling
b980fba 
On second thought this sort of special should be handled in Samba's
VFS rather than in ZFS.
@ghost ghost force-pushed the truenas/zfs-2.0-release branch from 79bb891 to fe36b5b Compare May 3, 2021 19:36
@anodos325
Copy link
Author

@freqlabs Do you need me to rebase this?

@ghost
Copy link

ghost commented May 6, 2021

Yes, needs to be rebased on truenas/zfs-2.1-release. I'll close this PR.

@ghost ghost closed this May 6, 2021
ixhamza pushed a commit that referenced this pull request Mar 20, 2023
@rob-wing
FreeBSD: don't verify recycled vnode for zfs control directory
28251d8 
Under certain loads, the following panic is hit:

    panic: page fault
    KDB: stack backtrace:
    #0 0xffffffff805db025 at kdb_backtrace+0x65
    #1 0xffffffff8058e86f at vpanic+0x17f
    #2 0xffffffff8058e6e3 at panic+0x43
    #3 0xffffffff808adc15 at trap_fatal+0x385
    #4 0xffffffff808adc6f at trap_pfault+0x4f
    #5 0xffffffff80886da8 at calltrap+0x8
    #6 0xffffffff80669186 at vgonel+0x186
    #7 0xffffffff80669841 at vgone+0x31
    #8 0xffffffff8065806d at vfs_hash_insert+0x26d
    #9 0xffffffff81a39069 at sfs_vgetx+0x149
    #10 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4
    #11 0xffffffff8065a28c at lookup+0x45c
    #12 0xffffffff806594b9 at namei+0x259
    #13 0xffffffff80676a33 at kern_statat+0xf3
    #14 0xffffffff8067712f at sys_fstatat+0x2f
    #15 0xffffffff808ae50c at amd64_syscall+0x10c
    #16 0xffffffff808876bb at fast_syscall_common+0xf8

The page fault occurs because vgonel() will call VOP_CLOSE() for active
vnodes. For this reason, define vop_close for zfsctl_ops_snapshot. While
here, define vop_open for consistency.

After adding the necessary vop, the bug progresses to the following
panic:

    panic: VERIFY3(vrecycle(vp) == 1) failed (0 == 1)
    cpuid = 17
    KDB: stack backtrace:
    #0 0xffffffff805e29c5 at kdb_backtrace+0x65
    #1 0xffffffff8059620f at vpanic+0x17f
    #2 0xffffffff81a27f4a at spl_panic+0x3a
    #3 0xffffffff81a3a4d0 at zfsctl_snapshot_inactive+0x40
    #4 0xffffffff8066fdee at vinactivef+0xde
    #5 0xffffffff80670b8a at vgonel+0x1ea
    #6 0xffffffff806711e1 at vgone+0x31
    #7 0xffffffff8065fa0d at vfs_hash_insert+0x26d
    #8 0xffffffff81a39069 at sfs_vgetx+0x149
    #9 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4
    #10 0xffffffff80661c2c at lookup+0x45c
    #11 0xffffffff80660e59 at namei+0x259
    #12 0xffffffff8067e3d3 at kern_statat+0xf3
    #13 0xffffffff8067eacf at sys_fstatat+0x2f
    #14 0xffffffff808b5ecc at amd64_syscall+0x10c
    #15 0xffffffff8088f07b at fast_syscall_common+0xf8

This is caused by a race condition that can occur when allocating a new
vnode and adding that vnode to the vfs hash. If the newly created vnode
loses the race when being inserted into the vfs hash, it will not be
recycled as its usecount is greater than zero, hitting the above
assertion.

Fix this by dropping the assertion.

FreeBSD-issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252700
Reviewed-by: Andriy Gapon <avg@FreeBSD.org>
Reviewed-by: Mateusz Guzik <mjguzik@gmail.com>
Reviewed-by: Alek Pinchuk <apinchuk@axcient.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Rob Wing <rob.wing@klarasystems.com>
Co-authored-by: Rob Wing <rob.wing@klarasystems.com>
Submitted-by: Klara, Inc.
Sponsored-by: rsync.net
Closes openzfs#14501
ixhamza pushed a commit that referenced this pull request Jun 12, 2023
@rob-wing @behlendorf
FreeBSD: don't verify recycled vnode for zfs control directory
f786232 
Under certain loads, the following panic is hit:

    panic: page fault
    KDB: stack backtrace:
    #0 0xffffffff805db025 at kdb_backtrace+0x65
    #1 0xffffffff8058e86f at vpanic+0x17f
    #2 0xffffffff8058e6e3 at panic+0x43
    #3 0xffffffff808adc15 at trap_fatal+0x385
    #4 0xffffffff808adc6f at trap_pfault+0x4f
    #5 0xffffffff80886da8 at calltrap+0x8
    #6 0xffffffff80669186 at vgonel+0x186
    #7 0xffffffff80669841 at vgone+0x31
    #8 0xffffffff8065806d at vfs_hash_insert+0x26d
    #9 0xffffffff81a39069 at sfs_vgetx+0x149
    #10 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4
    #11 0xffffffff8065a28c at lookup+0x45c
    #12 0xffffffff806594b9 at namei+0x259
    #13 0xffffffff80676a33 at kern_statat+0xf3
    #14 0xffffffff8067712f at sys_fstatat+0x2f
    #15 0xffffffff808ae50c at amd64_syscall+0x10c
    #16 0xffffffff808876bb at fast_syscall_common+0xf8

The page fault occurs because vgonel() will call VOP_CLOSE() for active
vnodes. For this reason, define vop_close for zfsctl_ops_snapshot. While
here, define vop_open for consistency.

After adding the necessary vop, the bug progresses to the following
panic:

    panic: VERIFY3(vrecycle(vp) == 1) failed (0 == 1)
    cpuid = 17
    KDB: stack backtrace:
    #0 0xffffffff805e29c5 at kdb_backtrace+0x65
    #1 0xffffffff8059620f at vpanic+0x17f
    #2 0xffffffff81a27f4a at spl_panic+0x3a
    #3 0xffffffff81a3a4d0 at zfsctl_snapshot_inactive+0x40
    #4 0xffffffff8066fdee at vinactivef+0xde
    #5 0xffffffff80670b8a at vgonel+0x1ea
    #6 0xffffffff806711e1 at vgone+0x31
    #7 0xffffffff8065fa0d at vfs_hash_insert+0x26d
    #8 0xffffffff81a39069 at sfs_vgetx+0x149
    #9 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4
    #10 0xffffffff80661c2c at lookup+0x45c
    #11 0xffffffff80660e59 at namei+0x259
    #12 0xffffffff8067e3d3 at kern_statat+0xf3
    #13 0xffffffff8067eacf at sys_fstatat+0x2f
    #14 0xffffffff808b5ecc at amd64_syscall+0x10c
    #15 0xffffffff8088f07b at fast_syscall_common+0xf8

This is caused by a race condition that can occur when allocating a new
vnode and adding that vnode to the vfs hash. If the newly created vnode
loses the race when being inserted into the vfs hash, it will not be
recycled as its usecount is greater than zero, hitting the above
assertion.

Fix this by dropping the assertion.

FreeBSD-issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252700
Reviewed-by: Andriy Gapon <avg@FreeBSD.org>
Reviewed-by: Mateusz Guzik <mjguzik@gmail.com>
Reviewed-by: Alek Pinchuk <apinchuk@axcient.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Rob Wing <rob.wing@klarasystems.com>
Co-authored-by: Rob Wing <rob.wing@klarasystems.com>
Submitted-by: Klara, Inc.
Sponsored-by: rsync.net
Closes openzfs#14501
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rick-mesta rick-mesta Awaiting requested review from rick-mesta

Assignees
No one assigned
Labels
WIP Work In Progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@anodos325 @mattmacy @sonicaj @kmoore134 @gmelikov