[BUG] Module not known on secureblue

[RoyalOughtness]

I've been doing additional testing. There are several points of breakage even in GDM:

1. I can still enter passwords even while locked out, and submitting passwords while locked out increases the time to unlock even if it's a valid password
2. The countdown no longer works. It shows a static string that doesn't change.

[RoyalOughtness]

Also, one note I forgot to mention is probably the most critical.

3. When I enter a valid password initially, it says "module not known" and I can't login at all

[RoyalOughtness]

FYI, I didn't forget about this and I'm still looking into it.

I think it might be related to this line which needs to be /usr/etc instead of /etc: secureblue/secureblue@1105f47#diff-b8072728d45e09c6d95ba6c907e87cd16aac92f9cafe491197b214720c18438eR8

@34N0 when you say you are unable to reproduce, do you mean on secureblue or in some other setup?

[RoyalOughtness]

scratch what I said above about the filepath, it's not correct. but I'm investigating regardless.

[RoyalOughtness]

I just set up a fresh silverblue VM, rebased to secureblue, ran yafti, set kargs, switched to the staging image, and I am able to repro "module not found".

If you have a secureblue VM as well, can you please run ujust check-local-overrides to make sure you don't have any modified files?

[34N0]

I'm on mobile right now but i think PAM looks for modules in '/lib64' which is a hyperlink to '/usr/lib64' on atomic images. The hyperlink might not exist until the first login?

If that is the case what if we would create the hyperlink in the build stage?

[RoyalOughtness]

@34N0 do you have that symlink in your secureblue VM? This still wouldn't explain how you ended up with the symlink but not me.

[RoyalOughtness]

also @34N0 are you able to repro on a completely fresh secureblue vm?

[34N0]

I never had this issue in my testing and i was unable to reproduce this as of now.

[RoyalOughtness]

@34N0 you tried a fresh secureblue vm?

[34N0]

@34N0 you tried a fresh secureblue vm?

yes. rebased to staging from a fresh silverblue vm.

[RoyalOughtness]

@34N0 and you have no local overrides? ujust check-local-overrides

[RoyalOughtness]

The reason I ask is that I suspect there might be some local overrides related to PAM being created by upstream silverblue at install time

[34N0]

@34N0 and you have no local overrides? ujust check-local-overrides

i checked but maybe i misunderstood the commands output. I reset the VM but i will try again soon an post the output in this thread.

[34N0]

The reason I ask is that I suspect there might be some local overrides related to PAM being created by upstream silverblue at install time

So this might be authselect related? authselect generates the pam service at install time. In secureblue we disable authselect and then edit the generated file via a script. If this is correct then maybe creating our own pam service file instead of editing it with a script resolves this issue.

[RoyalOughtness]

@34N0 that might be the answer. once you have the output for check-local-overrides it will give us more clues.

[34N0]

rebased from a fresh silverblue vm to the staging build and ran the diff command.

dev@fedora:~$ diff -r \
      --suppress-common-lines \
      --color="always" \
      --exclude "passwd*" \
      --exclude "group*" \
      --exclude="subgid*" \
      --exclude="subuid*" \
      --exclude="machine-id" \
      --exclude="adjtime" \
      --exclude="fstab" \
      --exclude="system-connections" \
      --exclude="shadow*" \
      --exclude="gshadow*" \
      --exclude="ssh_host*" \
      --exclude="cmdline" \
      --exclude="crypttab" \
      --exclude="hostname" \
      --exclude="localtime" \
      --exclude="locale*" \
      --exclude="*lock" \
      --exclude=".updated" \
      --exclude="*LOCK" \
      --exclude="vconsole*" \
      --exclude="00-keyboard.conf" \
      --exclude="grub" \
      --exclude="system.control*" \
      --exclude="cdi" \
      --exclude="default.target" \
      /usr/etc /etc 2>/dev/null | sed '/Binary\ files\ /d'
Only in /etc/authselect: authselect.conf
Only in /etc/authselect: dconf-db
Only in /etc/authselect: fingerprint-auth
Only in /etc/authselect: system-auth
Only in /etc/cups: subscriptions.conf.O
Only in /etc/sysconfig: anaconda
Only in /etc/sysconfig: network
Only in /usr/etc/systemd/system: cups.service
diff -r --suppress-common-lines '--color=always' --exclude 'passwd*' --exclude 'group*' '--exclude=subgid*' '--exclude=subuid*' '--exclude=machine-id' '--exclude=adjtime' '--exclude=fstab' '--exclude=system-connections' '--exclude=shadow*' '--exclude=gshadow*' '--exclude=ssh_host*' '--exclude=cmdline' '--exclude=crypttab' '--exclude=hostname' '--exclude=localtime' '--exclude=locale*' '--exclude=*lock' '--exclude=.updated' '--exclude=*LOCK' '--exclude=vconsole*' '--exclude=00-keyboard.conf' '--exclude=grub' '--exclude=system.control*' '--exclude=cdi' '--exclude=default.target' /usr/etc/yum.repos.d/_copr:copr.fedorainfracloud.org:phracek:PyCharm.repo /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:phracek:PyCharm.repo
9c9
< enabled=0
---
> enabled=1
diff -r --suppress-common-lines '--color=always' --exclude 'passwd*' --exclude 'group*' '--exclude=subgid*' '--exclude=subuid*' '--exclude=machine-id' '--exclude=adjtime' '--exclude=fstab' '--exclude=system-connections' '--exclude=shadow*' '--exclude=gshadow*' '--exclude=ssh_host*' '--exclude=cmdline' '--exclude=crypttab' '--exclude=hostname' '--exclude=localtime' '--exclude=locale*' '--exclude=*lock' '--exclude=.updated' '--exclude=*LOCK' '--exclude=vconsole*' '--exclude=00-keyboard.conf' '--exclude=grub' '--exclude=system.control*' '--exclude=cdi' '--exclude=default.target' /usr/etc/yum.repos.d/google-chrome.repo /etc/yum.repos.d/google-chrome.repo
7c7
< enabled=0
---
> enabled=1
diff -r --suppress-common-lines '--color=always' --exclude 'passwd*' --exclude 'group*' '--exclude=subgid*' '--exclude=subuid*' '--exclude=machine-id' '--exclude=adjtime' '--exclude=fstab' '--exclude=system-connections' '--exclude=shadow*' '--exclude=gshadow*' '--exclude=ssh_host*' '--exclude=cmdline' '--exclude=crypttab' '--exclude=hostname' '--exclude=localtime' '--exclude=locale*' '--exclude=*lock' '--exclude=.updated' '--exclude=*LOCK' '--exclude=vconsole*' '--exclude=00-keyboard.conf' '--exclude=grub' '--exclude=system.control*' '--exclude=cdi' '--exclude=default.target' /usr/etc/yum.repos.d/rpmfusion-nonfree-nvidia-driver.repo /etc/yum.repos.d/rpmfusion-nonfree-nvidia-driver.repo
5c5
< enabled=0
---
> enabled=1
diff -r --suppress-common-lines '--color=always' --exclude 'passwd*' --exclude 'group*' '--exclude=subgid*' '--exclude=subuid*' '--exclude=machine-id' '--exclude=adjtime' '--exclude=fstab' '--exclude=system-connections' '--exclude=shadow*' '--exclude=gshadow*' '--exclude=ssh_host*' '--exclude=cmdline' '--exclude=crypttab' '--exclude=hostname' '--exclude=localtime' '--exclude=locale*' '--exclude=*lock' '--exclude=.updated' '--exclude=*LOCK' '--exclude=vconsole*' '--exclude=00-keyboard.conf' '--exclude=grub' '--exclude=system.control*' '--exclude=cdi' '--exclude=default.target' /usr/etc/yum.repos.d/rpmfusion-nonfree-steam.repo /etc/yum.repos.d/rpmfusion-nonfree-steam.repo
5c5
< enabled=0
---
> enabled=1

There are the lines which are authselect related. But i do not quite understand their relevancy yet.

Only in /etc/authselect: authselect.conf
Only in /etc/authselect: dconf-db
Only in /etc/authselect: fingerprint-auth
Only in /etc/authselect: system-auth

[34N0]

The /etc/authselect/system-auth file is the template which gets copied by authselect to /etc/pam.d/system-auth. After that we disable authselect, and edit the file during build with our own script to add authramp.

I just read your comment again and you rebase from latest to staging an then get the not known error, correct? My best guess is still that this process doesn't work then since authramp is already disabled?

[RoyalOughtness]

@34N0 i rebased from Fedora's silverblue to staging, but I also copied over the relevant files from /usr to /usr/etc. Can you try doing the same and seeing if you can repro?

I'll try a fresh VM without doing that and see what I get.

[RoyalOughtness]

I'm perplexed.

I installed a fresh silverblue vm, then rebased to br-staging-39 of secureblue's silverblue, made no changes to files, and still, "module unknown"

[34N0]

am i rebasing to the wrong build? i always just rebase to staging not br-staging-39.

[RoyalOughtness]

@34N0 ohhhhh. yeah staging is an ancient tag from before the migration to bluebuild.

the new format for branch tags with bluebuild is br-${branchName}-${tag}

Okay this makes way more sense 😄

[RoyalOughtness]

let me know what you get with br-staging-39 and apologies for the confusion

[34N0]

haha ok this was my bad. i was able to reproduce this finally. This has to be an integration error in the current staging build rather than a flaw in the code. Leaving this issue open is fine though.

I'll try to build the staging build locally. Has a lot changed in the process since the bluebuild migration? When building locally i'll create a debug script which makes the build process more verbose. maybe i can spot the problem that way.

[RoyalOughtness]

This has to be an integration error in the current staging build rather than a flaw in the code.

I agree.

Has a lot changed in the process since the bluebuild migration?

Not particularly, except that there are indeed changes in bluebuild itself. This caused issues initially with setting file permissions. I wonder if this is a bluebuild issue as well.

locally i'll create a debug script which makes the build process more verbose. maybe i can spot the problem that way.

Thanks, I'll ask in the bluebuild discord to see if they have any ideas

[RoyalOughtness]

@34N0 I've determined this is likely not a secureblue-specific issue. I did the following:

1. install a fresh silverblue vm from fedora (not ublue or secureblue)
2. wget https://github.com/34N0/pam-authramp/releases/download/v0.9.99/pam-authramp-0.9.99-1.x86_64.rpm
3. rpm-ostree install pam-authramp-0.9.99-1.x86_64.rpm
4. ran sudo ./addauthramp.sh using the script from here
5. I then tried to run optoutauthselect.sh but it failed with:

$ sudo ./optoutauthselect.sh 
sudo: PAM account management error: Module is unknown
sudo: a password is required

So due to this bug I was unable to even complete the setup instructions because this error showed up after the addauthramp.sh step.

[34N0]

smart. hopefully this isn't a brakeage in the rpm spec.

i think using a script like this might be a bad move. it fuels cryptic errors like this. also i think the pam system configuration gets changed for major releases upstream.

also i've found another error in this approach. once a machine is rebased to a build using this script, rebasing back or to another build brakes the login mechanism.

[RoyalOughtness]

Yes, it also needs to be writing to /usr/etc/ instead of /etc for that reason.

However, there's still an underlying issue here. Please let me know once you have a recommendation for how to move forward.

[34N0]

update: so i am currently struggling a lot trying build and rebase locally. This might take some time...

[RoyalOughtness]

@34N0 no problem, please take your time!

Your efforts are greatly appreciated

[34N0]

i opened an issue in the bluebuild cli since i am currenty unable to build secureblue locally. blue-build/cli#154

[34N0]

This one was a hard one, but dae148e fixed it. I do not quite understand why but apparently for some authentication services to work, all modules loaded in that service need to expose a certain function, even fi they're not implemented. This does not affect all pam services which is why this did escape my integration tests and prior e2e testing.

I'll leave this open until i am absolutly sure this is fixed and i better understand why.

[RoyalOughtness]

@34N0 great news! thanks for digging into this