Merge pull request #2035 from matrix-org/travis/1.0/msc688-msc1227-la…

…zy-loading

Spec lazy-loading room members

api/client-server/definitions/room_event_filter.yaml

@@ -16,6 +16,20 @@ allOf:
 - type: object
   title: RoomEventFilter
   properties:
+    lazy_load_members:
+      type: boolean
+      description: |-
+        If ``true``, enables lazy-loading of membership events. See
+        `Lazy-loading room members <#lazy-loading-room-members>`_
+        for more information. Defaults to ``false``.
+    include_redundant_members:
+      type: boolean
+      description: |-
+        If ``true``, sends all membership events for all events, even if they have already
+        been sent to the client. Does not
+        apply unless ``lazy_load_members`` is ``true``. See
+        `Lazy-loading room members <#lazy-loading-room-members>`_
+        for more information. Defaults to ``false``.
     not_rooms:
       description: A list of room IDs to exclude. If this list is absent then no rooms
         are excluded. A matching room will be excluded even if it is listed in the ``'rooms'``

api/client-server/definitions/sync_filter.yaml

@@ -47,7 +47,7 @@ properties:
       not_rooms:
         description: A list of room IDs to exclude. If this list is absent then no rooms
           are excluded. A matching room will be excluded even if it is listed in the ``'rooms'``
-          filter. This filter is applied before the filters in ``ephemeral``, 
+          filter. This filter is applied before the filters in ``ephemeral``,
           ``state``, ``timeline`` or ``account_data``
         items:
           type: string
@@ -73,33 +73,6 @@ properties:
         allOf:
         - $ref: room_event_filter.yaml
         description: The state events to include for rooms.
-        properties:
-          lazy_load_members:
-            type: boolean
-            description: |-
-              If ``true``, the only ``m.room.member`` events returned in
-              the ``state`` section of the ``/sync`` response are those
-              which are definitely necessary for a client to display
-              the ``sender`` of the timeline events in that response.
-              If ``false``, ``m.room.member`` events are not filtered.
-              By default, servers should suppress duplicate redundant
-              lazy-loaded ``m.room.member`` events from being sent to a given
-              client across multiple calls to ``/sync``, given that most clients
-              cache membership events (see ``include_redundant_members``
-              to change this behaviour).
-          include_redundant_members:
-            type: boolean
-            description: |-
-              If ``true``, the ``state`` section of the ``/sync`` response will
-              always contain the ``m.room.member`` events required to display
-              the ``sender`` of the timeline events in that response, assuming
-              ``lazy_load_members`` is enabled. This means that redundant
-              duplicate member events may be returned across multiple calls to
-              ``/sync``. This is useful for naive clients who never track
-              membership data. If ``false``, duplicate ``m.room.member`` events
-              may be suppressed by the server across multiple calls to ``/sync``.
-              If ``lazy_load_members`` is ``false`` this field is ignored.
-
       timeline:
         allOf:
         - $ref: room_event_filter.yaml

api/client-server/event_context.yaml

@@ -34,6 +34,9 @@ paths:
         This API returns a number of events that happened just before and
         after the specified event. This allows clients to get the context
         surrounding an event.
+
+        *Note*: This endpoint supports lazy-loading of room member events. See `Filtering <#lazy-loading-room-members>`_
+        for more information.
       operationId: getEventContext
       security:
         - accessToken: []

api/client-server/message_pagination.yaml

@@ -33,6 +33,9 @@ paths:
       description: |-
         This API returns a list of message and state events for a room. It uses
         pagination query parameters to paginate history in the room.
+
+        *Note*: This endpoint supports lazy-loading of room member events. See `Filtering <#lazy-loading-room-members>`_
+        for more information.
       operationId: getRoomEvents
       security:
         - accessToken: []
@@ -111,6 +114,21 @@ paths:
                   type: object
                   title: RoomEvent
                   "$ref": "definitions/event-schemas/schema/core-event-schema/room_event.yaml"
+              state:
+                type: array
+                description: |-
+                  A list of state events relevant to showing the ``chunk``. For example, if
+                  ``lazy_load_members`` is enabled in the filter then this may contain
+                  the membership events for the senders of events in the ``chunk``.
+
+                  Unless ``include_redundant_members`` is ``true``, the server
+                  may remove membership events which would have already been
+                  sent to the client in prior calls to this endpoint, assuming
+                  the membership of those members has not changed.
+                items:
+                  type: object
+                  title: RoomStateEvent
+                  $ref: "definitions/event-schemas/schema/core-event-schema/state_event.yaml"
           examples:
             application/json: {
                 "start": "t47429-4392820_219380_26003_2265",

api/client-server/rooms.yaml

@@ -184,6 +184,44 @@ paths:
           description: The room to get the member events for.
           required: true
           x-example: "!636q39766251:example.com"
+        - in: query
+          name: at
+          type: string
+          description: |-
+            The point in time (pagination token) to return members for in the room.
+            This token can be obtained from a ``prev_batch`` token returned for
+            each room by the sync API. Defaults to the current state of the room,
+            as determined by the server.
+          x-example: "YWxsCgpOb25lLDM1ODcwOA"
+        # XXX: As mentioned in MSC1227, replacing `[not_]membership` with a JSON
+        # filter might be a better alternative.
+        # See https://github.com/matrix-org/matrix-doc/issues/1337
+        - in: query
+          name: membership
+          type: string
+          enum:
+            - join
+            - invite
+            - leave
+            - ban
+          description: |-
+            The kind of membership to filter for. Defaults to no filtering if
+            unspecified. When specified alongside ``not_membership``, the two
+            parameters create an 'or' condition: either the membership *is*
+            the same as ``membership`` **or** *is not* the same as ``not_membership``.
+          x-example: "join"
+        - in: query
+          name: not_membership
+          type: string
+          enum:
+            - join
+            - invite
+            - leave
+            - ban
+          description: |-
+            The kind of membership to exclude from the results. Defaults to no
+            filtering if unspecified.
+          x-example: leave
       security:
         - accessToken: []
       responses:

api/client-server/sync.yaml

@@ -34,6 +34,20 @@ paths:
         Clients use this API when they first log in to get an initial snapshot
         of the state on the server, and then continue to call this API to get
         incremental deltas to the state, and to receive new messages.
+
+        *Note*: This endpoint supports lazy-loading. See `Filtering <#filtering>`_
+        for more information. Lazy-loading members is only supported on a ``StateFilter``
+        for this endpoint. When lazy-loading is enabled, servers MUST include the
+        syncing user's own membership event when they join a room, or when the
+        full state of rooms is requested, to aid discovering the user's avatar &
+        displayname.
+
+        Like other members, the user's own membership event is eligible
+        for being considered redundant by the server. When a sync is ``limited``,
+        the server MUST return membership events for events in the gap
+        (between ``since`` and the start of the returned timeline), regardless
+        as to whether or not they are redundant.  This ensures that joins/leaves
+        and profile changes which occur during the gap are not lost.
       operationId: sync
       security:
         - accessToken: []
@@ -49,6 +63,8 @@ paths:
             requests. Creating a filter using the filter API is recommended for
             clients that reuse the same filter multiple times, for example in
             long poll requests.
+
+            See `Filtering <#filtering>`_ for more information.
           x-example: "66696p746572"
         - in: query
           name: since
@@ -125,6 +141,50 @@ paths:
                       title: Joined Room
                       type: object
                       properties:
+                        summary:
+                          title: RoomSummary
+                          type: object
+                          description: |-
+                            Information about the room which clients may need to
+                            correctly render it to users.
+                          properties:
+                            "m.heroes":
+                              type: array
+                              description: |-
+                                The users which can be used to generate a room name
+                                if the room does not have one. Required if the room's
+                                ``m.room.name`` or ``m.room.canonical_alias`` state events
+                                are unset or empty.
+
+                                This should be the first 5 members of the room, ordered
+                                by stream ordering, which are joined or invited. The
+                                list must never include the client's own user ID. When
+                                no joined or invited members are available, this should
+                                consist of the banned and left users. More than 5 members
+                                may be provided, however less than 5 should only be provided
+                                when there are less than 5 members to represent.
+
+                                When lazy-loading room members is enabled, the membership
+                                events for the heroes MUST be included in the ``state``,
+                                unless they are redundant. When the list of users changes,
+                                the server notifies the client by sending a fresh list of
+                                heroes. If there are no changes since the last sync, this
+                                field may be omitted.
+                              items:
+                                type: string
+                            "m.joined_member_count":
+                              type: integer
+                              description: |-
+                                 The number of users with ``membership`` of ``join``,
+                                 including the client's own user ID. If this field has
+                                 not changed since the last sync, it may be omitted.
+                                 Required otherwise.
+                            "m.invited_member_count":
+                              type: integer
+                              description: |-
+                                 The number of users with ``membership`` of ``invite``.
+                                 If this field has not changed since the last sync, it
+                                 may be omitted. Required otherwise.
                         state:
                           title: State
                           type: object
@@ -305,6 +365,14 @@ paths:
                 "rooms": {
                   "join": {
                     "!726s6s6q:example.com": {
+                      "summary": {
+                        "m.heroes": [
+                          "@alice:example.com",
+                          "@bob:example.com"
+                        ],
+                        "m.joined_member_count": 2,
+                        "m.invited_member_count": 0
+                      },
                       "state": {
                         "events": [
                           {

changelogs/client_server/newsfragments/2035.feature

@@ -0,0 +1 @@
+Add the option to lazy-load room members for increased client performance.

specification/client_server_api.rst

@@ -1317,6 +1317,66 @@ Filters can be created on the server and can be passed as as a parameter to APIs
 which return events. These filters alter the data returned from those APIs.
 Not all APIs accept filters.
 
+Lazy-loading room members
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Membership events often take significant resources for clients to track. In an
+effort to reduce the number of resources used, clients can enable "lazy-loading"
+for room members. By doing this, servers will attempt to only send membership events
+which are relevant to the client.
+
+It is important to understand that lazy-loading is not intended to be a
+perfect optimisation, and that it may not be practical for the server to
+calculate precisely which membership events are relevant to the client.  As a
+result, it is valid for the server to send redundant membership events to the
+client to ease implementation, although such redundancy should be minimised
+where possible to conserve bandwidth.
+
+In terms of filters, lazy-loading is enabled by enabling ``lazy_load_members``
+on a ``RoomEventFilter`` (or a ``StateFilter`` in the case of ``/sync`` only).
+When enabled, lazy-loading aware endpoints (see below) will only include
+membership events for the ``sender`` of events being included in the response.
+For example, if a client makes a ``/sync`` request with lazy-loading enabled,
+the server will only return membership events for the ``sender`` of events in
+the timeline, not all members of a room.
+
+When processing a sequence of events (e.g. by looping on ``/sync`` or
+paginating ``/messages``), it is common for blocks of events in the sequence
+to share a similar set of senders.  Rather than responses in the sequence
+sending duplicate membership events for these senders to the client, the
+server MAY assume that clients will remember membership events they have
+already been sent, and choose to skip sending membership events for members
+whose membership has not changed.  These are called 'redundant membership
+events'. Clients may request that redundant membership events are always
+included in responses by setting ``include_redundant_members`` to true in the
+filter.
+
+The expected pattern for using lazy-loading is currently:
+
+* Client performs an initial /sync with lazy-loading enabled, and receives
+  only the membership events which relate to the senders of the events it
+  receives.
+* Clients which support display-name tab-completion or other operations which
+  require rapid access to all members in a room should call /members for the
+  currently selected room, with an ``?at`` parameter set to the /sync
+  response's from token. The member list for the room is then maintained by
+  the state in subsequent incremental /sync responses.
+* Clients which do not support tab-completion may instead pull in profiles for
+  arbitrary users (e.g. read receipts, typing notifications) on demand by
+  querying the room state or ``/profile``.
+
+.. TODO-spec
+  This implies that GET /state should also take an ``?at`` param
+
+The current endpoints which support lazy-loading room members are:
+
+* |/sync|_
+* |/rooms/<room_id>/messages|_
+* |/rooms/{roomId}/context/{eventId}|_
+
+API endpoints
+~~~~~~~~~~~~~
+
 {{filter_cs_http_api}}
 
 Events