Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

lazyload aware /messages #3589

Merged
merged 73 commits into from
Aug 16, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
7b7fd27
untested attempt at deduplicating lazy-loaded members
ara4n Jun 3, 2018
c341d81
Merge branch 'develop' into matthew/remove_redundant_lazy_members
ara4n Jun 10, 2018
f7bd5da
add include_redundant_members filter option & make it work
ara4n Jun 10, 2018
589e5aa
merge and apply isort
ara4n Jul 19, 2018
8e66dd1
merge in #2970
ara4n Jul 19, 2018
a08b37b
fix bad merge
ara4n Jul 19, 2018
7362e6c
make /context lazyload & filter aware
ara4n Jul 20, 2018
cd28d2f
speed up /members and add at= and membership params
ara4n Jul 20, 2018
a17f0b6
make it work
ara4n Jul 20, 2018
9ba6ef2
Merge branch 'matthew/lazy_load_apis' into matthew/members_at
ara4n Jul 20, 2018
c6117fa
make it work
ara4n Jul 20, 2018
8f1585d
make filtering work
ara4n Jul 21, 2018
42308c0
initial cut at a room summary API
ara4n Jul 23, 2018
0beeecf
remove debug log
ara4n Jul 23, 2018
63ce31b
namespace the summary fields correctly
ara4n Jul 23, 2018
c8cbede
fix key typo
bwindels Jul 23, 2018
f9c3c26
Merge branch 'matthew/filter_members' into matthew/remove_redundant_l…
ara4n Jul 23, 2018
c2870ab
Merge branch 'matthew/remove_redundant_lazy_members' into matthew/laz…
ara4n Jul 23, 2018
ffb7a4c
Merge branch 'matthew/lazy_load_apis' into matthew/members_at
ara4n Jul 23, 2018
c1c6ff1
Merge branch 'matthew/members_at' into matthew/room_summary
ara4n Jul 23, 2018
7d99b0e
changelog
ara4n Jul 23, 2018
cd27a77
changelog
ara4n Jul 23, 2018
4018a6d
changelog
ara4n Jul 23, 2018
c238a88
changelog
ara4n Jul 23, 2018
e1daddd
first cut at lazyload aware /messages
ara4n Jul 23, 2018
dd2c0e1
make it work
ara4n Jul 23, 2018
40e61d5
changelog
ara4n Jul 23, 2018
1ba3683
return the correct counts & self-exclude from heros
ara4n Jul 24, 2018
f129390
spell heroes correctly
ara4n Jul 24, 2018
bfbebec
Merge branch 'matthew/room_summary' into matthew/lazy_load_messages_api
ara4n Jul 24, 2018
4c2edd1
remember to filter state events
ara4n Jul 24, 2018
e61071a
fix heroes definition to match MSC
ara4n Jul 24, 2018
d32e5f8
Merge branch 'matthew/filter_members' into matthew/remove_redundant_l…
ara4n Jul 24, 2018
238f750
deduplicating redundant members via event_id rather than mxid
ara4n Jul 24, 2018
2a79e1a
Merge branch 'matthew/remove_redundant_lazy_members' into matthew/laz…
ara4n Jul 24, 2018
48f3e43
Merge branch 'matthew/lazy_load_apis' into matthew/members_at
ara4n Jul 24, 2018
7cc81ed
Merge branch 'matthew/members_at' into matthew/room_summary
ara4n Jul 24, 2018
96d1f3c
Merge branch 'matthew/room_summary' into matthew/lazy_load_messages_api
ara4n Jul 24, 2018
08af91d
fix merge fail
ara4n Jul 24, 2018
e76254b
Merge branch 'matthew/members_at' into matthew/room_summary
ara4n Jul 24, 2018
4e33d06
Merge branch 'matthew/room_summary' into matthew/lazy_load_messages_api
ara4n Jul 24, 2018
58529df
Merge branch 'matthew/room_summary' of github.com:matrix-org/synapse …
erikjohnston Jul 27, 2018
9637ae4
Merge branch 'matthew/lazy_load_messages_api' of git+ssh://github.com…
ara4n Jul 27, 2018
d5b1842
fix missing import
ara4n Jul 27, 2018
e952368
incorporate review
ara4n Jul 30, 2018
423c512
add null check for state
ara4n Aug 7, 2018
2d9c062
Merge branch 'develop' into matthew/members_at
ara4n Aug 12, 2018
eebee08
convert /members?at= to take a stream token
ara4n Aug 12, 2018
5df2c36
Merge branch 'develop' into matthew/room_summary
ara4n Aug 12, 2018
d0c0d72
incorporate all the review feedback
ara4n Aug 12, 2018
b327e07
fix use of get_recent_event_ids_for_room
ara4n Aug 12, 2018
c6fd400
Merge branch 'matthew/room_summary' into matthew/lazy_load_messages_api
ara4n Aug 12, 2018
9e4acb7
Merge branch 'matthew/members_at' into matthew/room_summary
ara4n Aug 12, 2018
e235005
add cap neg to /versions for LL
ara4n Aug 12, 2018
db333cc
fix english
ara4n Aug 12, 2018
32bf4fa
return early rather than big if blocks
ara4n Aug 14, 2018
859ad35
incorporate PR review
ara4n Aug 14, 2018
ab31a44
Merge branch 'matthew/members_at' into matthew/lazy_load_messages_api
ara4n Aug 14, 2018
dd4498f
Merge branch 'develop' into matthew/members_at
ara4n Aug 14, 2018
53cce23
Merge branch 'matthew/members_at' into matthew/lazy_load_messages_api
ara4n Aug 14, 2018
65d4521
don't remove old versions from /versions for now
ara4n Aug 15, 2018
fe9ed88
Merge branch 'matthew/room_summary' into matthew/lazy_load_messages_api
ara4n Aug 15, 2018
217d5dd
Merge branch 'develop' into matthew/members_at
ara4n Aug 15, 2018
8442478
Merge branch 'matthew/members_at' into matthew/room_summary
ara4n Aug 15, 2018
0cbe5a2
Merge branch 'matthew/room_summary' into matthew/lazy_load_messages_api
ara4n Aug 15, 2018
f5189b9
remove incorrectly reintroduced method
richvdh Aug 15, 2018
0d5770d
cleanups
richvdh Aug 15, 2018
c2d9c96
Merge branch 'matthew/members_at' into matthew/room_summary
richvdh Aug 15, 2018
dc085c0
Merge remote-tracking branch 'origin/develop' into matthew/room_summary
richvdh Aug 15, 2018
7306074
remove spurious changelogs
richvdh Aug 15, 2018
c3cdc21
factor out get_lazy_loaded_members_cache
ara4n Aug 15, 2018
33fdfa7
Merge branch 'matthew/room_summary' into matthew/lazy_load_messages_api
ara4n Aug 15, 2018
62d8ce7
Merge branch 'develop' into matthew/lazy_load_messages_api
richvdh Aug 16, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/3589.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add lazy-loading support to /messages as per MSC1227
35 changes: 34 additions & 1 deletion synapse/handlers/pagination.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from twisted.internet import defer
from twisted.python.failure import Failure

from synapse.api.constants import Membership
from synapse.api.constants import EventTypes, Membership
from synapse.api.errors import SynapseError
from synapse.events.utils import serialize_event
from synapse.types import RoomStreamToken
Expand Down Expand Up @@ -251,6 +251,33 @@ def get_messages(self, requester, room_id=None, pagin_config=None,
is_peeking=(member_event_id is None),
)

state = None
if event_filter and event_filter.lazy_load_members():
# TODO: remove redundant members

types = [
(EventTypes.Member, state_key)
for state_key in set(
event.sender # FIXME: we also care about invite targets etc.
for event in events
)
]

state_ids = yield self.store.get_state_ids_for_event(
events[0].event_id, types=types,
)

if state_ids:
state = yield self.store.get_events(list(state_ids.values()))

if state:
state = yield filter_events_for_client(
self.store,
user_id,
state.values(),
is_peeking=(member_event_id is None),
)

time_now = self.clock.time_msec()

chunk = {
Expand All @@ -262,4 +289,10 @@ def get_messages(self, requester, room_id=None, pagin_config=None,
"end": next_token.to_string(),
}

if state:
chunk["state"] = [
serialize_event(e, time_now, as_client_event)
for e in state
]

defer.returnValue(chunk)
13 changes: 12 additions & 1 deletion synapse/rest/client/versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,22 @@ class VersionsRestServlet(RestServlet):
def on_GET(self, request):
return (200, {
"versions": [
# XXX: at some point we need to decide whether we need to include
# the previous version numbers, given we've defined r0.3.0 to be
# backwards compatible with r0.2.0. But need to check how
# conscientious we've been in compatibility, and decide whether the
# middle number is the major revision when at 0.X.Y (as opposed to
# X.Y.Z). And we need to decide whether it's fair to make clients
# parse the version string to figure out what's going on.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For documentation purposes in the hopes that it becomes uncovered:

#960 originally added 0.2.0 as well as 0.1.0 after a question asking if it should be added. Not a lot of descriptions for why, however.

"r0.0.1",
"r0.1.0",
"r0.2.0",
"r0.3.0",
]
],
# as per MSC1497:
"unstable_features": {
"m.lazy_load_members": True,
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trailing comma would be nice...

})


Expand Down