Many unread and unreadable rooms existing since compression

[telmich]

After using https://github.com/matrix-org/rust-synapse-compress-state we see the following behaviour:

• rooms that do not have new unread messages, are marked as unread
• when selecting these rooms in element-web, the rooms do not show any content
• When trying to access previous messages the error "failed to load timeline position" is raised

I am raising this issue here, as others running synapse might run into it.

I am currently trying to find out whether any kind of "matrix-db-fsck" exists that checks the integrity. Any pointers are welcome. I have raised the issue in the synapse-admins room as well.

[clokep]

I've had this moved to the compress repo since it seems related to running that tool.

[telmich]

Messages from the element-web console I see:

Could not find event $C-DUQVMF9fQJR4IVDL-jkgh9u9yJwT8EjTArFBp7JD8

In another room that exhibts this behaviour when trying to access a message by clicking "back on the timeline" button:

Error loading timeline panel at $zsMAnO1IfZj3a3IBCn1yl-yViFZll4vBh9zyHmd4llc: M_FORBIDDEN: You don't have permission to access that event.

[telmich]

Additionally some unencrypted rooms show notification symbols when there are no new messages.

[clokep]

It was pointed out that this might be matrix-org/synapse#6779, which is then made worse by using the state compressor. Maybe the query in matrix-org/synapse#6779 (comment) could be run to see if that is related.

[telmich]

I just checked the query mentioned in matrix-org/synapse#6779 (comment) and it does contain 0 rows.

What I see is that mostly 1 user account seems to be affected and a lot of private chats. Leaving a room and rejoining (even for unencrypted) does not restore the ability to see messages from before.

[telmich]

Something I also notice right now is that many of the unread/unreadable direct chats are actually listed below rooms instead of "people"

[telmich]

I was wondering whether anyone has a pointer to something like a "state fsck" that I could run to debug the situation?

[telmich]

It seems that this bug is still present in the current implementation. After continuing to apply the synapse_auto_compressor while synapse was offline, other members of the ungleich.ch homeserver reported afterwards the access denied to events.