Users joining a private room within a space via "restricted" join rule have no way of accessing (encrypted) history

[ara4n]

Steps to reproduce

1. Create a space
2. Create a private room in that space
3. Confirm that history visibility is set to shared
4. Set its join rules to be 'restricted' such that anyone in that space can join without an invite
5. Send some messages in the room
6. As a new user, join the space, and join the private room
7. Observe that you have no way of seeing the past history

Outcome

What did you expect?

If the room's history visibility is shared, newly joined users expect to be able to read history.

What happened instead?

Newly joined users have no way of reading history (unless they send keyshare requests to the original sending devices, but that will require those devices to still exist and be online and running in order to service the requests). Therefore the history will be suppressed.

Thoughts

This is a major problem for knowledge exchange within private spaces, as in something like Slack or Rocket.Chat you'd expect to see history when dipping into a channel in a given workspace. Some people workaround this by not enabling encryption on private rooms (ugh), but given the default behaviour is to encrypt private rooms, this ends up with a really bad experience of valuable history/context/knowledge being invisible to new members.

Another workaround is to manually export chat history and share as ZIPs within the room when new people join, but this is obviously a horrible hack.

Possible solutions

1. You could relax the requirement for which users can share keys with new members: e.g. admins could be allowed to do so, even if they didn't send the original message.. but it still requires their devices to be online. One mitigation could be to always have an admin bot online to manage a space (could also provide moderation, audit etc), but this then becomes a highly valuable target given it will amass E2EE keys.
2. You could send all keys for all join-restricted rooms within a space when a user is invited to a space - even if they haven't joined the room yet. This could be a lot of key traffic for conversations you don't care about, however - literally every key for every bit of scrollback the new user might ever want to read. Also, clients in general communicating within a room in a space would need to send megolm keys to every other user in that space, not just the room. This doesn't scale.
3. You could provide a social way for users to request keys off each other (One Click to join all subspaces #2286), but this is prone to social attacks.

I think the best bet might be option 2, but it's not ideal; other suggestions welcome.

[ara4n]

@BillCarsonFr points out that another option could be to store the keys for the rooms in the space in an online key backup, and then gossip the key for that backup to users who are invited to the space. This obviously defeats any post-compromise security, but given the whole point here is to share history enthusiastically with members of the space, PCS is pretty irrelevant for this use case.

[BillCarsonFr]

Rough description:

It would use the same API as for room_keys/
But it's scoped to a room (space), so if you are not member of the space a query to /rooms/{spaceId}/room_keys/** would 403 (forbidden).

A space admin would then create a new symetric backup version signed by him and store the versionas state event in the space room. The signature part might be new for symetric(?) / Or the signature could be in the state event instead of in auth_data

Then when someone invites a user to the space he then gossip the symetric backup key (encrypted m.secret.sendto device -4S sharing extension-). It could be possible to send secret request (TBD requirement to fulfil)

When a user joins a room restricted to a space, he can check for a backup on that space using the version in state event. The user can check the backup signature and decide to trust it (i.e signed by a space admin, eventually could also require that we have verified the admin / or tofu)
If trusted the user can start backing up keys, as well as downloading for this room /rooms/{spaceId}/room_keys/keys/{roomId}

[ara4n]

The only problem i can see with using online backup for this purpose is that it might freak out deployments who are allergic to online backup (given the fact it stores all their keys behind a single secret - e.g. Tchap). For them, we could perhaps do the "send all keys for all rooms to everyone in the space" trick (option 2 above)... or just say "sorry, no history in these rooms unless your deployment supports online backup"

[thawn]

At the very least, the option "let users see past history" should only be available on setups and rooms where this setting has an effect.

[BillCarsonFr]

Added a proposal to solve this by requesting manually inspired by the knocking join rule
#750

[anoadragon453]

Lots of discussion in an internal Element room, which may be useful to extract conclusions from: https://matrix.to/#/!ZCWYLfAKXolsPWObuZ:matrix.org/$0EQlzqgHoTmJzE8qxHhzzop1LihT7-ArlZG4MIdBtAw?via=matrix.org&via=element.io&via=one.ems.host