zfs ignores umask for files created with O_TMPFILE unless acltype=posixacl

[gyakovlev]

System information

Type	Version/Name
Distribution Name	gentoo
Distribution Version	current
Linux Kernel	5.1.14-gentoo
Architecture	amd64
ZFS Version	0.8.1-r0-gentoo
SPL Version	0.8.1-r0-gentoo

Describe the problem you're observing

zfs seems to be ignoring umask if files created with O_TMPFILE unless acltype=posixacl

detailed explanation here https://bugs.gentoo.org/686142

Describe how to reproduce the problem

minimal reproducer (credits: @trofi)

// $ gcc a.c -o a && ./a

#define _GNU_SOURCE /* for O_TMPFILE */

#include <fcntl.h>
#include <unistd.h>

#include <sys/types.h>
#include <sys/stat.h>

#include <errno.h>
#include <limits.h>
#include <stdio.h>
#include <string.h>

static char probe_name[] = "file-for-b686142";

static void probe_umask(void) {
    mode_t current_umask = umask(0);
    umask(current_umask);

    printf("current umask = %#o\n", (unsigned)current_umask);
}

static void probe_open(void) {
    struct stat fs;
    struct stat s;
    int fd;
    int e;
    char path[PATH_MAX];
    mode_t mode = 0777;

    memset(&fs, 0, sizeof(fs));
    memset(&s, 0, sizeof(s));

    // create target file
    fd = open(".", O_TMPFILE | O_RDWR, mode);
    fstat(fd, &fs);
    snprintf(path, PATH_MAX,  "/proc/self/fd/%d", fd);
    linkat(AT_FDCWD, path, AT_FDCWD, probe_name, AT_SYMLINK_FOLLOW);
    close (fd);

    stat(probe_name, &s);
    unlink(probe_name);

    printf("fstat(): in_mode=%#o, resulting_mode=%#o\n", (unsigned)mode, (unsigned)fs.st_mode);
    printf("stat():  in_mode=%#o, resulting_mode=%#o\n", (unsigned)mode, (unsigned)s.st_mode);
}

static void probe_open_modes(void) {
    probe_umask();
    probe_open();
}

int main() {
    // test defaults
    probe_open_modes();

    // test restricted masks
    umask(0077);
    probe_open_modes();
}

localhost /zp # zfs set acltype=posixacl zp
localhost /zp # gcc a.c -o a && ./a
current umask = 022
fstat(): in_mode=0777, resulting_mode=0100755
stat():  in_mode=0777, resulting_mode=0100755
current umask = 077
fstat(): in_mode=0777, resulting_mode=0100700
stat():  in_mode=0777, resulting_mode=0100700

localhost /zp # zfs set acltype=noacl zp
localhost /zp # gcc a.c -o a && ./a
current umask = 022
fstat(): in_mode=0777, resulting_mode=0100777
stat():  in_mode=0777, resulting_mode=0100777
current umask = 077
fstat(): in_mode=0777, resulting_mode=0100777
stat():  in_mode=0777, resulting_mode=0100777

as you can see with posixacl it honors umask and does not with noacl

Include any warning/errors/backtraces from the system logs

[gyakovlev]

sorry pressed open too soon

[trofi]

Extra note: proximate cause is lack of umask application for noacl case. MusicPlayerDaemon/MPD#558 (comment) has example fixes for other filesystems (don't know if it's a correct approach or generic vfs layer would be better place to enforce umask handling).