Room history sometimes not visible even when historical room keys are shared.

[uhoreg]

When inviting a user to an E2EE room with history visibility set to allow the new user to view historical messages, we now share (some) decryption keys with the invited user. However, even after doing so, sometimes the new user will see a blank room when they join. They can view the messages after switching to a different room and then switching back.

This is due to matrix-org/matrix-react-sdk#3881, which filters out UTDs from the timeline if they are from before the user joined the room. Thus, if the invited user's Element tries to display the timeline before it receives/processes the keys, it will not try to re-display the undecryptable messages.

I think that we can fix this by modifying https://github.com/matrix-org/matrix-react-sdk/pull/3881/files so that, instead of removing the undecryptable event tiles from the timeline, it just hides them or leaves them blank. When the keys come in, it re-tries the decryption (using the normal decryption re-try mechanism), and if the decryption is successful, then it should back-paginate a few events, and either hide these events and stop back-paginating (if undecryptable), or show them back-paginate more (if decryptable).

Note: I think that this exceeds my React knowledge, so hopefully someone else can pick it up.

---

You can't see earlier messages

Encrypted messages before this point are unavailable

[t3chguy]

So I cannot reproduce this, I synthesised it by creating a public encrypted room R, sending some messages from user A, joining R from user B, then issuing mxMatrixClientPeg.matrixClient.sendSharedHistoryKeys($R, [$B]) on user A and immediately user B sees the remaining history.

Screen.Recording.2021-10-12.at.10.10.03.mov

There have been some changes to the TimelinePanel recently due to Threading, maybe they made this path happier accidentally? Has someone seen this in the wild on develop recently?

[djedirahu]

So I cannot reproduce this, I synthesised it by creating a public encrypted room R, sending some messages from user A, joining R from user B, then issuing mxMatrixClientPeg.matrixClient.sendSharedHistoryKeys($R, [$B]) on user A and immediately user B sees the remaining history.

Screen.Recording.2021-10-12.at.10.10.03.mov
There have been some changes to the TimelinePanel recently due to Threading, maybe they made this path happier accidentally? Has someone seen this in the wild on develop recently?

Any reason why this function, or simply sharing keys on invite would break and no longer continue to function as expected @t3chguy ? I know this issue has been closed but I am experiencing just that on certain existing rooms and it's becoming quite a problem for our community in it's infant stages bringing new people into the space who are greeted with empty rooms. If I try to replicate it by creating a new space and populating it with e2ee rooms, it functions fine?

[t3chguy]

@djedirahu the share keys on invite feature requires the megolm sessions to be flagged as shareable, which only Element Web & Desktop do right now, so megolm sessions created by other clients won't be shareable yet.

[djedirahu]

@djedirahu the share keys on invite feature requires the megolm sessions to be flagged as shareable, which only Element Web & Desktop do right now, so megolm sessions created by other clients won't be shareable yet.

Any way to manually set that flag for an existing room?

[t3chguy]

The flag isn't on the room, it's on each megolm session, megolm sessions are created by each sending device periodically.

[djedirahu]

The flag isn't on the room, it's on each megolm session, megolm sessions are created by each sending device periodically.

Thanks for the clarification, excuse my gaps in understanding. In this case all of the invites have been sent via web or desktop (or utilising the dev console via the web app as above), to no avail. Nothing indicates that it shouldn't send the keys as intended on invite. Was wondering if there was anything else to investigate if it still fails to function. As I said creating new e2ee rooms seems to solve it, but for our existing rooms that are populated with content and members it seems to only function periodically or not at all.

[djedirahu]

Oh hang on so if the messages are sent via ios or android they won't appear, as the keys are only shared from sessions created with the flag enabled? The majority of our users are posting from ios or android - so will not appear. Thought they were tied exclusively to the invite but it actually has to do with each megolm session on a per message basis... Any idea if this will be changed for ios/android in the near future?

@djedirahu the share keys on invite feature requires the megolm sessions to be flagged as shareable, which only Element Web & Desktop do right now, so megolm sessions created by other clients won't be shareable yet.

[HarHarLinks]

I think I am hitting this. I'm invited into an e2ee room which has some history. Element web shows a blank timeline apart from my invite & join event. I can scroll up into a blank area.

At the same time, Element android does show the history, but messages are either "Unable to decrypt: OLM" or "Unable to decrypt: The sender's device has not sent us the keys for this message."

[ara4n]

This happens pretty much every time I invite someone to an E2EE room, despite keysharing happening. Either the history pops into view when they send a message, or they have to bg and then fg the room to make it visible. It causes a massive WTF despite us having gone to the trouble of implementing keysharing.

(We might also want to doublecheck that the /invite command correctly keyshares too, as opposed to using the UI)

[lampholder]

Unless it's changed since I checked wtih @uhoreg /invite does not implement keyshares.

[t3chguy]

Confirmed, only the InviteDialog calls MatrixClient::sendSharedHistoryKeys

[turt2live]

(put as P1 in the web team board because a duplicate issue was P1 - I am assuming someone else will re-triage as needed)

Edit: except not because I closed the wrong issue

[novocaine]

I briefly saw this today in the Offsite 2022 room (which I'd joined yesterday), the message about not being able to display further history was displayed after the room very slowly loaded a large number of join events when I switched to the room.

I saw the tile "You can't see earlier messages" but I didn't take note of which subtitle it had.

Yesterday, I actually was able to see the history, so it was a bit of a surprise that the history had since disappeared! I then switched rooms and the history magically reappeared.

So I'm left wondering if this is not just a bug associated with initial joining, but there are other things going on as well, given I initially joined the room yesterday.

[MadLittleMods]

I've seen this many times in the main timeline.

But just had a reproduction in the thread panel for the first time. Normally, the thread panel is just blank until messages load.

[mguentner]

I also experience this. Funnily enough I can see / browse previous messages in element-desktop when I search for the content in the search. 😄

[hazecodeio]

The issues has returned with the recent roll-out :( New members can't see the history unless they have been at least invited before any new messages in the room.

"** Unable to decrypt: DecryptionError: This message was sent when we were not a member of the room. **"

Element version: 1.11.69
Crypto version: Rust SDK 0.7.0 (068a0af), Vodozemac 0.6.0