Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outstanding CI issue #434

Open
mrc0mmand opened this issue Mar 6, 2024 · 0 comments
Open

Outstanding CI issue #434

mrc0mmand opened this issue Mar 6, 2024 · 0 comments

Comments

@mrc0mmand
Copy link
Member

mrc0mmand commented Mar 6, 2024

(I'm opening this mainly as a reminder to myself, so I don't debug the same stuff twice.)

In #433 CI started failing with:

 120/298 test-fs-util                                       FAIL              0.05s   killed by signal 6 SIGABRT
>>> SYSTEMD_KBD_MODEL_MAP=/build/src/locale/kbd-model-map SYSTEMD_LANGUAGE_FALLBACK_MAP=/build/src/locale/language-fallback-map MALLOC_PERTURB_=236 PATH=/build/build:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /build/build/test-fs-util
 ✀  
stderr:
Assertion 'touch_file(a, false, test_mtime, test_uid, test_gid, 0640) >= 0' failed at ../src/test/test-fs-util.c:549, function test_touch_file(). Aborting.

The fail itself is caused by touch_file() calling chmod() on a symlink, which then returns EOPNOTSUPP. I suspect this is something that has changed in later kernels (and in this case it affects CI, because it runs in a docker container on Ubuntu Jammy). On C8S kernel this seems to work, but only on tmpfs:

# cat main.c 
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/types.h>

int main(int argc, char *argv[]) {
        int fd, r;
        char fdpath[64];

        if (argc < 2)
                return 1;

        fd = open(argv[1], O_PATH|O_CLOEXEC|O_NOFOLLOW);
        if (fd < 0) {
                fprintf(stderr, "open() failed: %m\n");
                return 1;
        }

        sprintf(fdpath, "/proc/self/fd/%i", fd);
        r = chmod(fdpath, 0640);
        printf("chmod(%s)=%d (%d)\n", fdpath, r, errno);

        return r;
}
# mount -t tmpfs tmpfs /tmp
# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
# gcc -o main main.c -D_GNU_SOURCE
# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
# ./main /var/tmp/dangling 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /var/tmp/symlink 
chmod(/proc/self/fd/3)=-1 (95)

And since the test uses /dev/shm/ for the test files (which is either tmpfs or some other form of tmpfs), it works.

On newer kernels all attempts to call chmod() on a symlink fail:

# uname -r
6.7.4-200.fc39.x86_64
# ./main /tmp/dangling 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /tmp/symlink 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /var/tmp/dangling 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /var/tmp/symlink 
chmod(/proc/self/fd/3)=-1 (95)

The fix on the touch_file() side would be to call fchmod_and_chown() instead. Unfortunately, our fchmod_and_chown() in RHEL 8 is not clever enough and still does chmod() on a symlink, suffering from the same issue. For that we'd need 2dbb7e9 (+ commits before and after that change) to make it work properly, and that seems way too risky given how far in we're in RHEL 8 cycle.

So, when the need arises, I think it would be safer to just skip the offending test case, i.e.:

a = strjoina(p, "/lnk");
assert_se(symlink("target", a) >= 0);
assert_se(touch_file(a, false, test_mtime, test_uid, test_gid, 0640) >= 0);
assert_se(lstat(a, &st) >= 0);
assert_se(st.st_uid == test_uid);
assert_se(st.st_gid == test_gid);
assert_se(S_ISLNK(st.st_mode));
assert_se((st.st_mode & 0777) == 0640);
assert_se(timespec_load(&st.st_mtim) == test_mtime);

when running in GH Actions (or docker, whatever).

mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in a
container. This should work around this issue in GH Actions, but still
run the test on C8S/RHEL8 machines.

See: redhat-plumbers#434

RHEL-only
mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in a
container. This should work around this issue in GH Actions, but still
run the test on C8S/RHEL8 machines.

See: redhat-plumbers#434

RHEL-only
mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in a
container. This should work around this issue in GH Actions, but still
run the test on C8S/RHEL8 machines.

See: redhat-plumbers#434

RHEL-only
mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in GH
Actions. To do that we have to jump through a couple of hoops, since
RHEL 8 systemd can't detect docker. Oh well.

See: redhat-plumbers#434

RHEL-only
mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in GH
Actions. To do that we have to jump through a couple of hoops, since
RHEL 8 systemd can't detect docker. Oh well.

See: redhat-plumbers#434

RHEL-only
mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in GH
Actions. To do that we have to jump through a couple of hoops, since
RHEL 8 systemd can't detect docker. Oh well.

See: redhat-plumbers#434

RHEL-only
mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in GH
Actions. To do that we have to jump through a couple of hoops, since
RHEL 8 systemd can't detect docker. Oh well.

See: redhat-plumbers#434

RHEL-only
mrc0mmand added a commit to mrc0mmand/rhel-8 that referenced this issue Mar 13, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in GH
Actions. To do that we have to jump through a couple of hoops, since
RHEL 8 systemd can't detect docker. Oh well.

See: redhat-plumbers#434

RHEL-only
jamacku pushed a commit that referenced this issue Jun 25, 2024
Our (RHEL 8) touch_file() is not clever enough and does chmod() on a
symlink, which fails with EOPNOTSUPP on newer kernels. This is not an
issue on the RHEL 8 kernel, where doing chmod() on a symlink works
(albeit only on tmpfs) but in GH Actions we run in a container, and with
the underlying kernel doing chmod() on a symlink fails even on tmpfs:

RHEL 8:
~# mount -t tmpfs tmpfs /tmp
~# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
~# gcc -o main main.c -D_GNU_SOURCE
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Newer kernel:
~# uname -r
6.7.4-200.fc39.x86_64
~# ./main /tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/dangling
chmod(/proc/self/fd/3)=-1 (95)
~# ./main /var/tmp/symlink
chmod(/proc/self/fd/3)=-1 (95)

Backporting the necessary patches would be way too risky so late in the
RHEL 8 cycle, so let's just skip the offending test when running in GH
Actions. To do that we have to jump through a couple of hoops, since
RHEL 8 systemd can't detect docker. Oh well.

See: #434

RHEL-only
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant