From 8cc34dfc550bf36ef7a71ac0da08a1ad8fd0046e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 30 Aug 2024 10:24:06 +0100 Subject: [PATCH 1/3] Simplified sliding sync --- proposals/4186-simplified-sliding-sync.md | 345 ++++++++++++++++++++++ 1 file changed, 345 insertions(+) create mode 100644 proposals/4186-simplified-sliding-sync.md diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md new file mode 100644 index 0000000000..41b4382e24 --- /dev/null +++ b/proposals/4186-simplified-sliding-sync.md @@ -0,0 +1,345 @@ +# MSC4186: Simplified Sliding Sync + +The current `/sync` endpoint scales badly as the number of rooms on an account increases. It scales badly because all +rooms are returned to the client, incremental syncs are unbounded and slow down based on how long the user has been +offline, and clients cannot opt-out of a large amount of extraneous data such as receipts. On large accounts with +thousands of rooms, the initial sync operation can take tens of minutes to perform. This significantly delays the +initial login to Matrix clients, and also makes incremental sync very heavy when resuming after any significant pause in +usage. + +Note: this is a “simplified” version of the sliding sync API proposed in +[MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575), based on paring back that API based +on real world use cases and usages. + + +# Goals + +This improved `/sync` mechanism has a number of goals: + +- Sync time should be independent of the number of rooms you are in. +- Time from launch to confident usability should be as low as possible. +- Time from login on existing accounts to usability should be as low as possible. +- Bandwidth should be minimized. +- Support lazy-loading of things like read receipts (and avoid sending unnecessary data to the client) +- Support informing the client when room state changes from under it, due to state resolution. +- Clients should be able to work correctly without ever syncing in the full set of rooms they’re in. +- Don’t incremental sync rooms you don’t care about. +- Servers should not need to store all past since tokens. If a since token has been discarded we should gracefully + degrade to initial sync. + +These goals shaped the design of this proposal. + + +# Proposal + +The core differences between sync v2 and simplified sliding sync are: + +- The server initially only sends the most recent N rooms to the client (where N is specified by the lcient), which then + can paginate in older rooms in subsequent requests +- The client can configure which information the server will return for different sets of rooms (e.g. a smaller timeline + limit for older rooms). +- The client can filter what rooms it is interested in +- The client can maintain multiple sync loops (with some caveats) + +The basic operation is similar between sync v2 and simplified sliding sync: both use long-polling with tokens to fetch +updates from the server. I.e., the basic operation of both APIs is to do an “initial” request and then repeatedly call +the API supplying the token returned in the previous response in the subsequent “incremental” request. + + +## Lists and room subscriptions + +The core component of a sliding sync request is “lists”, which specify what information to return about which rooms. +Each list specifies some filters on rooms (e.g. ignore spaces), the range of filtered rooms to select (e.g. the most +recent 20 filtered rooms), and the config for the data to return for those rooms (e.g. the required state, timeline +limit, etc). The order of rooms is always done based on when the server received the most recent event for the room. + +The client can also specify config for specific rooms if it has their room ID, these are known as room subscriptions. + +Multiple lists and subscriptions can be specified in a request. If a room matches multiple lists/subscriptions then the +config is “combined” to be the superset of all configs (e.g. take the maximum timeline limit). See below for the exact +algorithm. + +The server tracks what data has been sent to the client in which rooms. If a room matches a list or subscription that +hasn’t been sent down before, then the server will respond with the full metadata about the room indicated by `initial: +true`. If a room stops matching a list (i.e. it falls out of range) then no further updates will be sent until it starts +matching a list again, at which point the missing updates (limited by the `timeline_limit`) will be sent down. However, +as clients are now expected to paginate all rooms in the room list in the background (in order to correctly order and +search them), the act of a room falling out of range is a temporary edge-case. + + +## Pagination + +Pagination is achieved by the client increasing the ranges of one (or more) lists. + +For example an initial request might have a list called `all_rooms` specifying a range of `0..20` in the initial +request, and the server will respond with the top 20 rooms (by most recently updated). On the second request the client +may change the range to `0..100`, at which point the server will respond with the top 100 rooms that either a) weren’t +sent down in the first request, or b) have updates since the first request. + +Clients can increase and decrease the ranges as they see fit. A common approach would be to start with a small window +and grow that until the range covers all the rooms. After some threshold of the app being offline it may reduce the +range back down and incrementally grow it again. This allows for ensuring that a limited amount of data is requested at +once, to improve response times. + + +## Connections + +Clients can have multiple “connections” (i.e. sync loops) with the server, so long as each connection has a different +`conn_id` set in the request. + +Clients must only have a single request in-flight at any time per connection (clients can have multiple connections by +specifying a unique `conn_id`). If a client needs to send another request before receiving a response to an in-flight +request (e.g. for retries or to change parameters) the client *must* cancel the in-flight request and *not* process any +response it receives for it. + +In particular, a client must use the returned `pos` value in a response as the `since` param in exactly one request that +the client will process the response for. Clients must be careful to ensure that when processing a response any new +requests use the new `pos`, and any in-flight requests using an old `pos` are canceled. + +The server cannot assume that a client has received a response until it receives a new request with the `since` token +set to the `pos` it returned in the response. The server must ensure that any per-connection state it tracks correctly +handles receiving multiple requests with the same `since` token (e.g. the client retries the request or decides to +cancel and resend a request with different parameters). + +A server may decide to “expire” connections, either to free resources or because the server thinks it would be faster +for the client to start from scratch (e.g. because there are many updates to send down). This is done by responding with +a 400 HTTP status and an error code of `M_UNKNOWN_POS`. + + +## List configuration + +**TODO**, these are the same as in [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): + +- Required state format +- The filters +- Lazy loading of members +- Combining room config + + +## Room config changes + +When a room comes in and out of different lists or subscriptions, the effective `timeline_limit` and `required_state` +parameters may change. This section outlines how the server should handle these cases. + +If the `timeline_limit` *increases* then the server *may* choose to send down more historic data. This is to support the +ability to get more history for certain rooms, e.g. when subscribing to the currently visible rooms in the list to +precache their history. This is done by setting `unstable_expanded_timeline` to true and sending down the last N events +(this may include events that have already been sent down). The server may choose not to do this if it believes it has +already sent down the appropriate number of events. + +If new entries are added to `required_state` then the server must send down matching current state events. + + +## Extensions + +We anticipate that as more features land in Matrix, different kinds of data will also want to be synced to clients. Sync +v2 did not have any first-class support to opt-in to new data. Sliding Sync does have support for this via "extensions". +Extensions also allow this proposal to be broken up into more manageable sections. Extensions are requested by the +client in a dedicated extensions block. + +In an effort to reduce the size of this proposal, extensions will be done in separate MSCs. There will be extensions +for: + +- To Device Messaging \- MSC3885 +- End-to-End Encryption \- MSC3884 +- Typing Notifications \- MSC3961 +- Receipts \- MSC3960 +- Presence \- presence in sync v2: spec +- Account Data \- account\_data in sync v2: MSC3959 +- Threads + +**TODO** explain how these interact with the room lists, this is the same as in +[MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) + +## Request format + +```javascript +{ + "conn_id": "", // Client chosen ID of the connection, c.f. "Connections" + + // The set of room lists + "lists": { + // An arbitrary string which the client is using to refer to this list for this connection. + "": { + + // Sliding window ranges, c.f. Lists and room subscriptions + "ranges": [[0, 10]], + + // Filters to apply to the list. + "filters": { + // Flag which only returns rooms present (or not) in the DM section of account data. + // If unset, both DM rooms and non-DM rooms are returned. If false, only non-DM rooms + // are returned. If true, only DM rooms are returned. + "is_dm": true|false|null, + + // Flag which only returns rooms which have an `m.room.encryption` state event. If unset, + // both encrypted and unencrypted rooms are returned. If false, only unencrypted rooms + // are returned. If true, only encrypted rooms are returned. + "is_encrypted": true|false|null, + + // Flag which only returns rooms which have an `m.room.encryption` state event. If unset, + // both encrypted and unencrypted rooms are returned. If false, only unencrypted rooms + // are returned. If true, only encrypted rooms are returned. + "is_invite": true|false|null, + + // If specified, only rooms where the `m.room.create` event has a `type` matching one + // of the strings in this array will be returned. If this field is unset, all rooms are + // returned regardless of type. This can be used to get the initial set of spaces for an account. + // For rooms which do not have a room type, use 'null' to include them. + "room_types": [ ... ], + + // Same as "room_types" but inverted. This can be used to filter out spaces from the room list. + // If a type is in both room_types and not_room_types, then not_room_types wins and they are + // not included in the result. + "not_room_types": [ ... ], + }, + + // The maximum number of timeline events to return per response. + "timeline_limit": 10, + + // Required state for each room returned. An array of event type and state key tuples. + // Elements in this array are ORd together to produce the final set of state events + // to return. One unique exception is when you request all state events via ["*", "*"]. When used, + // all state events are returned by default, and additional entries FILTER OUT the returned set + // of state events. These additional entries cannot use '*' themselves. + // For example, ["*", "*"], ["m.room.member", "@alice:example.com"] will _exclude_ every m.room.member + // event _except_ for @alice:example.com, and include every other state event. + // In addition, ["*", "*"], ["m.space.child", "*"] is an error, the m.space.child filter is not + // required as it would have been returned anyway. + "required_state": [ ... ], + } + }, + + // The set of room subscriptions + "room_subscriptions": { + // The key is the room to subscribe to. + "!foo:example.com": { + // These have the same meaning as in `lists` section + "timeline_limit": 10, + "required_state": [ ... ], + } + }, + + // c.f. "Extensions" + "extensions": { + } +} +``` + + +## Response format + +```javascript +{ + // The position to use as the `since` token in the next sliding sync request. + // c.f. Connections. + "pos": "", + + // Information about the lists supplied in the request. + "lists": { + // Matches the list name supplied by the client in the request + "" { + // The total number of rooms that match the list's filter. + "count": 1234, + } + }, + + // Aggregated rooms from lists and room subscriptions. There will be one entry per room, even if + // the room appears in multiple lists and/or room subscriptions. + "rooms": { + "!foo:example.com": { + // The room name, if one exists. Only sent initially and when it changes. + "name": str|null, + // The room avatar, if one exists. Only sent initially and when it changes. + "avatar_url": str|null, + // The "heroes" for the room, if there is no room name. Only sent initially and when it changes. + "heroes": [ + {"user_id":"@alice:example.com","displayname":"Alice","avatar_url":"mxc://..."}, + ], + + // Flag which is set when this is the first time the server is sending this data on this connection. + // When set the client must replace any stored metadata for the room with the new data. In + // particular, the state must be replaced with the state in `required_state`. + "initial": true|null, + + // Same as in sync v2. Indicates whether there are more events to fetch than those in the timeline. + "limited:" true|null, + // Indicates if we have "expanded" the timeline due to the timeline_limit changing, c.f. Room config + // changes above. + "unstable_expanded_timeline": true|null, + // The list of events, sorted least to most recent. + "timeline": [ ... ], + // The current state of the room as a list of events + "required_state": [ ... ], + // The number of timeline events which have just occurred and are not historical. + // The last N events are 'live' and should be treated as such. + "num_live": 1, + // Same as sync v2, passed to `/messages` to fetch more past events. + "prev_batch": "...", + + // For invites this is the stripped state of the room at the time of invite + "invite_state": [ .. ], + + // For knocks this is the stripped state of the room at time of knock + "knock_state": [ .. ], + + // Whether the room is a DM room. + "is_dm": true|null, + + // An opaque integer that can be used to sort the rooms by "Bump Stamp" + "bump_stamp": 1, + + // These are the same as sync v2. + "joined_count": 1, + "invited_count": 1, + "notification_count": 1, + "highlight_count": 1, + } + }, + + "extensions": { + }, +} +``` + + +# Alternatives / changes + +There are a number of potential changes that we could make. + +## Pagination + +In practice, having the client specify the ranges to use for the lists is often sub-optimal. The client generally wants +to have the sync request return as quickly as possible, but it doesn't know how much data the server has to return and +so whether to increase or decrease the range. + +An alternative is for the client to specify a `page_size`, where the server sends down at most `page_size` number of +rooms. If there are more rooms to send to the client (beyond `page_size`), then the client can request to "paginate" in +these missed updates in subsequent updates. + +Since this would require client side changes, this should be explored in a separate MSC. + +## Timeline event trickling + +If the `timeline_limit` is increased then the server will send down historic data (c.f. "Room config changes"), which +allows the clients to easily preload more history in recent rooms. + +This mechanism is fiddly to implement, and ends up resending down events that we have previously sent to the client. + +A simpler alternative is to use `/messages` to fetch the history. This has two main problems: 1) clients generally want +to preload history for multiple rooms at once, and 2) `/messages` can be slow if it tries to backfill over federation. + +We could implement a bulk `/messages` endpoint, where the client would specify multiple rooms and `prev_batch` tokens. +We can also add a flag to disable attempting to backfill over pagination (to match the behaviour of the sync timeline). + +## `required_state` response format + +The format of returned state in `required_state` is a list of events. This does now allow the server to indicate if a +"state reset" has happened which removed an entry from the state entirely (rather than it being replaced with another +event). + +This is particularly problematic if the user gets "state reset" out of the room, where the server has no mechanism to +indicate to the client that the user has effectively left the room (the server has no leave event to return). + +We may want to allow special entries in the `required_state` list of the form +`{"type": .., "state_key": .., content: null}` to indicate that the state entry has been removed. From 456db3b9d3b936457c9e780e439e4e6cd30f28a9 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 30 Aug 2024 10:52:28 +0100 Subject: [PATCH 2/3] Add example usage --- proposals/4186-simplified-sliding-sync.md | 27 +++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 41b4382e24..9c6d7379dd 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -302,6 +302,33 @@ for: } ``` +# Example usage + +This section gives an example of how a client can use this API (roughly based on how Element X currently uses the API). + +When the app starts up it configures a single list with a range of `[0, 19]` (to get the top 20 rooms) and a +`timeline_limit` of 1. This returns quickly with the top 20 rooms (or just the changes in the top 20 rooms if a token +was specified). + +The client then increases the range (in the next request) to `[0, 99]`, which will return the next 80 rooms. The server +may sort the rooms differently than they are returned by the server (e.g. they may ignore reactions for sorting +purposes). + +The client can use room subscriptions, with a `timeline_limit` of 20, to preload history for the top rooms. This means +that if the user clicks on one of the top rooms the app can immediately display a screens worth of history. (An +alternative would be to have a second list with a static range of `[0, 19]` and a `timeline_limit` of 20. The downside) + +The client can keep increasing the list range in increments to pull in the full list of rooms. The client uses the +returned `count` for the list to know when to stop expanding the list. + +The client *may* decided to reduce the range back to `[0, 19]` (and then subsequently incrementally expand the range), +this can be done. + +When the client is expecting a fast response (e.g. while expanding the lists), it should set the `timeout` parameter to +0 to ensure the server doesn't block waiting for new data. This can easily happen if the app starts and sends the first +request with a `since` parameter, if the client shows a spinner but doesn't set a timeout then the request may take a +long time to return (if there were no updates to return). + # Alternatives / changes From 6fe4ba72da515f3e473be42a57a7a32f03e303f8 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 30 Aug 2024 10:59:21 +0100 Subject: [PATCH 3/3] Add security and unstable prefix --- proposals/4186-simplified-sliding-sync.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 9c6d7379dd..275512a7cc 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -370,3 +370,15 @@ indicate to the client that the user has effectively left the room (the server h We may want to allow special entries in the `required_state` list of the form `{"type": .., "state_key": .., content: null}` to indicate that the state entry has been removed. + + +# Security considerations + +Care must be taken, as with sync v2, to ensure that only the data that the user is authorized to see is returned in the +response. + + +# Unstable prefix + +The unstable URL for simplified sliding sync is `/org.matrix.simplified_msc3575/sync`. The flag in `/versions` is +`org.matrix.simplified_msc3575`.