The authentication key/token which is stored by KWallet gets changed automatically for no apparent reason

[nasko]

Summary

The authentication key/token which is stored by KWallet gets changed automatically for no apparent reason. When this happens, all SFDX CLI commands return this error:

Failed to decipher auth data. reason: Unsupported state or unable to authenticate data.

When at this point I execute sf org list, I'm getting this output:

(D) DEV-HUB-PRIME myuser-devhub@mydomain.com 00D2E0000014737UAA AuthDecryptError

I can employ several alternative methods to see that the auth token/key has been changed:

• By navigating to the Secret Service -> Passwords -> 'salesforce.com' in KWalletManager
• By executing this command in the terminal: $ secret-tool lookup domain sfdx
• By executing this command in the terminal: kwallet-query -f 'Secret Service' -r "'salesforce.com'" kdewallet

Steps To Reproduce:

I haven't yet identified any definitive steps to reproduce this issue. It just happens randomly.

To start clean, I'm performing these steps:

• Log out from any scratch- or non-scratch org via sf org logout, such that sf org list returns no orgs at all.
• Delete the currently active 'salesforce.com' secret from KWallet, by navigating to: KWalletManager -> Secret Service -> Passwords -> 'salesforce.com'
• Log in to a Dev Hub with sf org login
• Retrieve and save in a text file the newly generated secret by executing: kwallet-query -f 'Secret Service' -r "'salesforce.com'" kdewallet
• Continue executing various SF CLI commands related to my ongoing development tasks.
• At some random point, the 'salesforce.com' secret will get swapped for a newly generated one. The timing for this random moment is hard to predict - sometimes it takes several days before this happens, but occasionally this would happen after a few command executions.
• If at this point I restore the previously saved secret token in KWalletManager, SFDX CLI will continue working without any faults.

Expected result

SFDX should not swap the existing secret with a newly generated one.

Actual result

At some random point, the 'salesforce.com' secret will get swapped for a newly generated one.

System Information

• OS: Linux

• Distribution: Arch Linux

• Desktop environment: KDE Plasma 5.27

• Shell: bash

• The gnome-keyring optional dependency is not installed.

• The Use KWallet for the Secret Service interface option is checked in KDE Wallet system settings.

• sf version --verbose --json output:

{
  "cliVersion": "@salesforce/cli/1.80.0",
  "architecture": "linux-x64",
  "nodeVersion": "node-v18.15.0",
  "osVersion": "Linux 6.3.3-arch1-1",
  "shell": "bash",
  "rootPath": "/opt/sfdx-cli/sf",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 2.2.0 (core)",
    "@oclif/plugin-commands 2.2.15 (core)",
    "@oclif/plugin-help 5.2.9 (core)",
    "@oclif/plugin-not-found 2.3.23 (core)",
    "@oclif/plugin-plugins 3.0.1 (core)",
    "@oclif/plugin-search 0.0.17 (core)",
    "@oclif/plugin-update 3.1.15 (core)",
    "@oclif/plugin-version 1.3.4 (core)",
    "@oclif/plugin-warn-if-update-available 2.0.37 (core)",
    "@oclif/plugin-which 2.2.20 (core)",
    "@salesforce/cli 1.80.0 (core)",
    "apex 2.2.19 (core)",
    "auth 2.7.15 (core)",
    "data 2.3.18 (core)",
    "deploy-retrieve 1.9.2 (core)",
    "info 2.6.13 (core)",
    "limits 2.3.15 (core)",
    "login 1.2.9 (core)",
    "org 2.9.1 (core)",
    "schema 2.3.10 (core)",
    "settings 1.4.8 (core)",
    "sobject 0.1.20 (core)",
    "source 2.10.9 (core)",
    "telemetry 2.2.0 (core)",
    "templates 55.4.15 (core)",
    "trust 2.4.18 (core)",
    "user 2.3.13 (core)"
  ]
}

• sfdx version --verbose --json output:

{
  "cliVersion": "sfdx-cli/7.202.7",
  "architecture": "linux-x64",
  "nodeVersion": "node-v18.15.0",
  "osVersion": "Linux 6.3.3-arch1-1",
  "shell": "bash",
  "rootPath": "/opt/sfdx-cli",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 2.2.0 (core)",
    "@oclif/plugin-commands 2.2.15 (core)",
    "@oclif/plugin-help 5.2.9 (core)",
    "@oclif/plugin-not-found 2.3.23 (core)",
    "@oclif/plugin-plugins 3.0.1 (core)",
    "@oclif/plugin-search 0.0.17 (core)",
    "@oclif/plugin-update 3.1.15 (core)",
    "@oclif/plugin-version 1.3.4 (core)",
    "@oclif/plugin-warn-if-update-available 2.0.37 (core)",
    "@oclif/plugin-which 2.2.20 (core)",
    "apex 2.2.19 (core)",
    "auth 2.7.15 (core)",
    "community 2.2.11 (core)",
    "custom-metadata 2.1.22 (core)",
    "data 2.3.18 (core)",
    "deploy-retrieve 1.9.2 (core)",
    "info 2.6.13 (core)",
    "limits 2.3.15 (core)",
    "org 2.9.1 (core)",
    "packaging 1.17.1 (core)",
    "schema 2.3.10 (core)",
    "settings 1.4.8 (core)",
    "signups 1.4.18 (core)",
    "source 2.10.9 (core)",
    "telemetry 2.2.0 (core)",
    "templates 55.4.15 (core)",
    "trust 2.4.18 (core)",
    "user 2.3.13 (core)",
    "sfdx-cli 7.202.7 (core)"
  ]
}

Additional information

Secret in KWallet Manager:

AuthDecryptError in terminal:

KWallet System settings:

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[mlaso]

+1

Same here on Manjaro + KDE (fresh install). For me was causing so much pain as I work on multiple sandboxes and orgs at the same time (testing some CI/CD stuff).

It was a pain to re-authenticate all the orgs. As a workaround at the moment, I have to back-up the secret value after successfully authenticating a bunch of orgs. When the issue comes up, I just restore the value to the previous one.

Every single auth method (web, device, auth url, etc) is affected by this issue.

[mshanemc]

one workaround would be to set SFDX_USE_GENERIC_UNIX_KEYCHAIN which will avoid using the KDE wallet for secret storage. Not sure if that's a good solution for you or not--it'll generate a key in the .sfdx folder and use that to encrypt secrets.

Otherwise, I'm not sure how we'd replicate this without a linux machine, KDE, etc. And it might turn out to be a KDE wallet issue.

If you want to try debugging this keyChain implementation stuff, the code starts around here: https://github.com/forcedotcom/sfdx-core/blob/1c42cf051cd98ae5811681e992c257785740f3a4/src/crypto/keyChain.ts#L47-L48

[nasko]

Thanks for the suggestion @mshanemc, but setting that env variable doesn't seem to affect the behavior:

• I logged out of all connected orgs
• Deleted the stored secret in KDE Wallet
• Set the SFDX_USE_GENERIC_UNIX_KEYCHAIN variable to true
• Killed the terminal, started it again and made sure that echo $SFDX_USE_GENERIC_UNIX_KEYCHAIN printed true
• Logged to a dev hub using sf org login web
• Observed that a new secret had been stored in KDE wallet
• Verified that tweaking the stored secret's value, followed by an sf org list command resulted in an AuthDecryptError error

[nasko]

I'm willing to try to track the execution flow within retrieveKeychain, but I'm not quite well versed with the SFDX CLI infrastructure. For example I found five separate instances of keyChain.js within my /opt/sfdx-cli folder. I added console.log() statements in all five, but can't see any output in STDOUT when I'm executing commands. If you could point me to the right direction to read up on debugging this stuff, I'd be grateful.
Thanks

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

[zacherytapp]

Hey - just coming to this to say I stumbled across this exact error (Nobara 39, KDE) , I had my default browser set to Google Chrome, when I restarted (before opening Google Chrome) - I confirmed I still had this error and then tried to authenticate using sf org login web and Chromium seemed to be the default vs. Google Chrome. I authenticated using that browser and haven't had an issue since.

[nasko]

Hey @zacherytapp! Thanks for your input!
In my case I don't think the issue is related to the default browser, as the defect is manifested even when I haven't executed the sf org login web command at all - even just creating several scratch orgs, followed by around 10 executions of the sf org list command would typically cause the error to manifest. Luckily I have a bash script dedicated to restoring the secret that I have previously stored in a text file: ~/.secret. Every time the issue is manifested I'd just execute secret-restore and that would resolve it. The thing is, I sometimes have to do this several times a day.

/usr/local/bin/secret-backup:

#!/bin/bash
kwallet-query -f 'Secret Service' -r "'salesforce.com'" kdewallet > ~/.secret

/usr/local/bin/secret-restore:

#!/bin/bash
kwallet-query -f 'Secret Service' -w "'salesforce.com'" kdewallet < ~/.secret

It's sad the SF CLI team obviously primarily use non-linux boxes, and would tend to assume that most developers are like them, so they consider Linux users' problems a minor concern. :-(

CC: @mshanemc

[Zosoled]

I just encountered this issue tonight. It was extremely frustrating and actually led to me using up my daily scratch org creation limit before I was ready to call it quits for the day. I hope Salesforce reconsiders fixing this issue.

[nasko]

@Zosoled I doubt it that would happen, to be honest.

Judging from how @mshanemc simply deferred debugging this to us, and couldn't even be bothered to provide any hints on how this can be debugged after I asked, speaks for itself.

@mshanemc said:

I'm not sure how we'd replicate this without a linux machine, KDE, etc

I mean, how hard could this be?! The fact that there are devs who use this under linux, and KDE Plasma is one of the major graphical shells, used by thousands of people, should be reason enough for Salesforce to spare the necessary resources to support it.

@Zosoled my advice would be to create these two one-line scripts that would allow you to backup- and restore the secret once this issue is manifested. This happens at least once every couple of days, but once I see the AuthDecryptError in my console, I'd just type: secret-restore and be with it. Then I'd simply repeat the last SF CLI command that failed.

[Zosoled]

one workaround would be to set SFDX_USE_GENERIC_UNIX_KEYCHAIN which will avoid using the KDE wallet for secret storage. Not sure if that's a good solution for you or not--it'll generate a key in the .sfdx folder and use that to encrypt secrets.

Looks like it is not SFDX_... but SF_..., as in SF_USE_GENERIC_UNIX_KEYCHAIN.
https://github.com/forcedotcom/sfdx-core/blob/1c42cf051cd98ae5811681e992c257785740f3a4/src/crypto/keyChain.ts#L26C53-L26C81

[Zosoled]

There are two attribute-value pairs being created by the use of secret-tool: user local and domain sfdx. Initially they are set to the same value from what I can tell, but maybe at some point they diverge. Is there a reason both are needed?

[Zosoled]

@mshanemc can this be reopened? It is still affecting my pipeline and is EXCEPTIONALLY aggravating when it destroys sf access to all my orgs.

[nasko]

@Zosoled Recently I gave the SF_USE_GENERIC_UNIX_KEYCHAIN another chance and found out that it does the job just fine.
One caveat is that if I set this environment variable in my ~/.bashrc file, it is available in my console sessions as expected, but is not available to the WebStorm IDE, which obviously sets its environment prior to my console session being ready. The workaround was to set the variable in a place that's evaluated even earlier in the boot sequence - /etc/environment.

Ever since I set this environment variable, Kdewallet is not used any more and I haven't experienced the AuthDecryptError since.

[Zosoled]

@nasko Thanks, I'll give that a go. It would still be nice if the issue was fixed, but I won't hold my breath.