Login not possible since upgrade from 1.50.2 to 1.51.0

[vmario89]

Hi,

i processed the update from 1.50.2 1.51.0 on our Ubuntu 20 Server. Before the update the server worked properly and logins were possible. But now the homesderver.log always says

Attempted to login as @vmario89:matrix.fablabchemnitz.de but they do not exist

<XForwardedForRequest at 0x7fcb68457850 method='POST' uri='/_matrix/client/r0/login' clientproto='HTTP/1.0' site='8008'> SynapseError: 403 - Invalid password

We cannot login by web or dekstop client anymore which is fatal for us.

What could it be?

What i already checked so far

• matrix-synapse is running and journalctl output looks normal to me
• psql db is up and running, select statements on the USERS table is fine and the users i try to login really exist
• registering new users is not possible too. The server says registration is disabled but this is not true in terms of homeserver.yaml ("SynapseError: 403 - Registration has been disabled")
  the federation is up and running too https://federationtester.matrix.org/api/report?server_name=matrix.fablabchemnitz.de

i dont see my question as pure support request. i think something might be broken.

Did something change with macaroons or signing keys?

[DMRobertson]

Hi @vmario89. That is very strange. A few questions:

• if you downgrade to 1.50.2, are you able to login?
• was there a database upgrade or other system upgrade? I.e. did anything apart from synapse change?

[reivilibre]

This might sound stupid, but are you sure the correct configuration file is being loaded? If you can't log in to existing accounts (wrong/different password hash pepper?) and Synapse is saying registration is disabled even though you don't think it is, something is either very wrong with the way configuration is loaded or somehow some configuration files have been mixed up.

Also: what's the installation method? (PyPI, Docker, matrix.org's Debian/Ubuntu Packages?)

[richvdh]

[DMRobertson]

@vmario89 did you manage to work out what was going on with your server? Are you able to login again, and if so, what changed? Many thanks!

[vmario89]

hi. sorry for being inconvient. i was totally frustrated when i closed this issue. i was able to fix it. the thing i did when destroying it: i installed the upgrade with apt and overwrote the homeserver.yaml with the DEFAULT file. interestingly absolutely no error was shown in any log except the posted one's in this issue...

i solved the issue simply by restoring homeserver.yaml from backup

finally this resulted in all users of our homeserver were logged out and some people lost their personal chat history because they didnt store their personal encryption passwords accordingly.

for now i will never use apt update && apt upgrade to perform upgrade

i will use that command to avoid failing at the homeserver.yaml ...
apt-get -y -o Dpkg::Options::=--force-confdef install --only-upgrade matrix-synapse-py3

[reivilibre]

Related: #11281 (we now advise you to create separate configuration override files so that you don't ever touch the one included by apt)
#8159 (we want to separate config docs from configuration itself)
#11238 (another issue about this problem of being asked to overwrite your installation)