EACCES when root opens user files with O_WRONLY|O_CREAT|O_APPEND

[bjaglin]

Issue Report

Bug

Container Linux Version

OK on 2023.5.0
KO on 2079.1.0
KO on 2107.0.0

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=2079.1.0
VERSION_ID=2079.1.0
BUILD_ID=2019-03-24-2042
PRETTY_NAME="Container Linux by CoreOS 2079.1.0 (Rhyolite)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"
COREOS_BOARD="amd64-usr"

OK on Ubuntu with the same linux kernel as 2079.1.0 (4.19.31)

Environment

EC2

Expected Behavior

root should be able to append to any file

Actual Behavior

root gets EACCES when appending to a file owned by another regular user

Reproduction Steps

core@sand-2154-1555061726 ~ $ echo foo > /tmp/bar
core@sand-2154-1555061726 ~ $ ls -l /tmp/bar
-rw-r--r--. 1 core core 4 Apr 12 16:18 /tmp/bar
core@sand-2154-1555061726 ~ $ sudo su       
sand-2154-1555061726 core # echo > /tmp/bar 
bash: /tmp/bar: Permission denied
sand-2154-1555061726 core # echo >> /tmp/bar 
bash: /tmp/bar: Permission denied
sand-2154-1555061726 core # cat /tmp/bar 
foo
sand-2154-1555061726 core # rm /tmp/bar 
sand-2154-1555061726 core # uname -a
Linux sand-2154-1555061726 4.19.31-coreos #1 SMP Sun Mar 24 20:06:59 -00 2019 x86_64 Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz GenuineIntel GNU/Linux

[ajeddeloh]

Confirmed I can reproduce this. Investigating. Doesn't look to be SELinux related since I can repro in permissive mode and I don't see any audit logs.

[ajeddeloh]

Looks like it doesn't affect the rootfs, but does affect tmpfs mounts. I tried with an xfs mount and that also worked. My gut says its a kernel issue but it seems weird that 1) it would have been outstanding a breaking for a long time. and 2) ubuntu's kernel of the same version does not exhibit the problem.

[ajeddeloh]

This was introduced by systemd/systemd@2732587. We're reverting for next beta (which will then promote to stable). In the mean time you can follow the steps here: systemd/systemd@2732587#diff-ff4e2dc4962dc25a1512353299992c8dR37

[ajeddeloh]

Closed via coreos/coreos-overlay#3638

[bjaglin]

Thanks for the insanely fast turnaround!

For the record - this breaking change was exposed on a MySQL container failing to start, without any sort of log. strace showed that the cause of this was that opening the error log file (owned by another user for some obscure reason) was denied.

$ strace mysqld --user=root
...
open("/tmp/mysql.err", O_WRONLY|O_CREAT|O_APPEND, 0666) = -1 EACCES (Permission denied)