This repository has been archived by the owner on Apr 26, 2024. It is now read-only.
Leaving room throws 500 internal error #7916
Labels
X-Needs-Info
This issue is blocked awaiting information from the reporter
z-bug
(Deprecated Label)
z-p2
(Deprecated Label)
Comments
Do you happen to know what those versions were? We have messed a bit with JSON stuff recently (see #7674). |
|
Looks like the relevant code lines have been changed (after v1.17.0): f460da6#diff-5a0797e349006661a23d6b420d7735d1 |
anoadragon453
added
z-bug
|
@anoadragon453 Ah, yeah I wonder if something came in as the wrong data type ("I have moved synapse twice via postgres dump. I don't know whether that has had an effect on the integrity of my data base.") If so, that PR should fix it. |
|
@nomelif Please let us know if this issue is resolved in the next Synapse release. You may also try running on the latest |
Looking at the git blame for nixpkgs, the previous version they distributed was 1.16.1 and the one before was 1.15.2. |
I'll let you know. I don't want to mess around with multiple versions of synapse if the case happens to be that my db is flaky, tho. |
I wonder if I could see what postgres returns somehow. |
|
Have updated my synapse to 1.18.0 (as it landed in nixpkgs-stable) but this is still broken. |
|
From journalctl: |
|
If my db is the problem, would there be a way to see that by messing with psql directly? |
|
This log message in particular seems interesting:
As it turns out, the TypeError message from simplejson is extremely misleading and actually means whatever was passed in here was not a So my hypothesis is that at least one row in the |
|
@nomelif can you check to see if there are any NULLs in your DB? |
|
No nulls: |
|
Stronger statement: there is no column with nulls in that table |
|
Interesting. Can you try: |
|
Only text |
|
For some reason trying to join a room also fails. I haven't managed to get a good error yet tho. |
|
Hmm, the last thing I can think of trying is |
|
|
Would there be a way to figure out the exact request synapse is making? |
|
You could turn up logging for SQL temporarily, though its very very verbose: |
|
I have installed the thing through nix and configured it statelessly. I don't really have writable config files |
|
I'll try to figure out if I can catch the queries on the postgres side |
Can't you change the log config in your nix config? |
|
should be possible: https://nixos.org/nixos/options.html#services.matrix-synapse.logconfig |
|
Quite frustrating when you're trying to clear your client out a bit and you just encounter this over and over:
I assume it's the same error mentioned here. |
this is likely to be a bad assumption. Please open a new issue and share the server logs. |
|
@nomelif are you still seeing this problem? there's likely little we can do to help unless you are able to adjust your logging settings. |
|
Closing this due to lack of response. |
|
Sorry, I admit I forgot about this until the issue was closed. I managed (I think) to put the debug flags on. Trying to leave the room leaves this log in systemd I admit I am not sure if I configured the thing correctly. I took the default log settings file and appended the few lines given by @erikjohnston |
these should be under a |
richvdh
added
the
X-Needs-Info
|
In the time between @richvdh 's last message, I tried to get rid of the rooms in the hopes of re-joining them. I used a command of the kind: This managed to hide the room from element, but it is in a state where I can not re-join it. I added the logger lines and tried to capture joining it: I am starting to ponder the benefits of simply nuking postgres and reinstalling everything. It seems fairly clear that there is some broken lingering state which I have myself partly amplified while trying to fix it. |
richvdh
removed
the
X-Needs-Info
|
sorry for the slow response time here.
A couple of things here. Firstly, I'm afraid it's not complete - although it shows the DEBUG lines now, it no longer shows the original error, so we can't tell what operation you were trying to do. Secondly:
This suggests that you have directed log output to stderr. This isn't a great idea because, by default, stderr is redirected to the logs - so you get a loop. The loop gets caught, but the resultant logging is very unclear. You can suppress the redirection of stderr with a In summary: it looks like your log config is somewhat bogus. If you upload the config file here, we might be able to offer advice as to what is wrong with it. |
richvdh
added
the
X-Needs-Info
|
I am afraid I have re-installed matrix with a wiped postgres. It works with the same configuration as before, so I am inclined to believe that something went awry while migrating it. Thank you for your time and sorry for wasting it. |
|
never mind, if it works now we'll consider that a win :) |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Description
Synapse throws a 500 internal error when I try to leave a room.
Steps to reproduce
This is only broken for me with the postmarketOS offtopic channel for some reason. I have moved synapse twice via postgres dump. I don't know whether that has had an effect on the integrity of my data base. I was trying to leave the room in the first place because new messages stopped being received (into that specific room, other rooms work). They were being received for some time (months) after the second migration though.
What I expected: room is left successfully
What happened: a 500 error is thrown, the room is not left.
I got a clear error from
journalctl:Version information
If not matrix.org:
This was broken for the two previous matrix versions from nixpkgs as well. I hoped that the updates would fix it.
nixpkgs.matrix-synapseThe text was updated successfully, but these errors were encountered: