MSC4108 implementation #17056

sandhose · 2024-04-05T16:30:41Z

This is a native implementation of MSC4108

It works by keeping the pushed content in an in-memory map. It means this is meant to be handled by the same worker.
This map has a fixed capacity, and entries are regularly evicted.

The capacity and various other knobs are hard-coded for now, and could be exposed in the config.
It also doesn't have any rate limiting, meaning you could force the expiration of other user's entries by spamming the creation of new request.
Both of those shortcomings could be handled in a separate pull request.

CLAassistant · 2024-04-10T15:01:56Z

All committers have signed the CLA.

hughns · 2024-04-11T14:29:12Z

synapse/http/server.py

@@ -909,7 +909,18 @@ def set_cors_headers(request: "SynapseRequest") -> None:
    request.setHeader(
        b"Access-Control-Allow-Methods", b"GET, HEAD, POST, PUT, DELETE, OPTIONS"
    )
-    if request.experimental_cors_msc3886:
+    if request.path is not None and request.path.startswith(
+        b"/_matrix/client/unstable/org.matrix.msc4108/rendezvous"


This path needs to match https://github.com/element-hq/synapse/pull/17056/files#diff-60b44b832bdef37a1bd6f9174c4e59ad6539a88363ad7292c4a48e9e556da4a7R81

Maybe it should be a shared constant?

I fixed the path but it isn't using a constant

synapse/http/server.py

This does not add the config bits, but does add the plumbing to set it from the Python size

hughns · 2024-04-18T11:00:04Z

synapse/rest/client/rendezvous.py

+        self._handler = hs.get_rendezvous_handler()
+
+    def on_POST(self, request: SynapseRequest) -> None:
+        self._handler.handle_post(request)


The POST endpoint should have rate limiting on it.

Ah, I forgot that you had put a note to do this in another PR.

anoadragon453

Just some small comments, nothing major. Thanks for adding tests, and love all the rust here!

rust/src/rendezvous/mod.rs

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

anoadragon453

This LGTM after #17122, thanks!

Issues to track follow-up work:

anoadragon453 · 2024-04-25T12:28:00Z

Linting tests were failing due to #17125. This PR has now been updated with latest develop, and now all CI checks will run.

@Fizzadar

No significant changes since 1.106.0rc1. - Send an email if the address is already bound to an user account. ([\#16819](element-hq/synapse#16819)) - Implement the rendezvous mechanism described by [MSC4108](matrix-org/matrix-spec-proposals#4108). ([\#17056](element-hq/synapse#17056)) - Support delegating the rendezvous mechanism described [MSC4108](matrix-org/matrix-spec-proposals#4108) to an external implementation. ([\#17086](element-hq/synapse#17086)) - Add validation to ensure that the `limit` parameter on `/publicRooms` is non-negative. ([\#16920](element-hq/synapse#16920)) - Return `400 M_NOT_JSON` upon receiving invalid JSON in query parameters across various client and admin endpoints, rather than an internal server error. ([\#16923](element-hq/synapse#16923)) - Make the CSAPI endpoint `/keys/device_signing/upload` idempotent. ([\#16943](element-hq/synapse#16943)) - Redact membership events if the user requested erasure upon deactivating. ([\#17076](element-hq/synapse#17076)) - Add a prompt in the contributing guide to manually configure icu4c. ([\#17069](element-hq/synapse#17069)) - Clarify what part of message retention is still experimental. ([\#17099](element-hq/synapse#17099)) - Use new receipts column to optimise receipt and push action SQL queries. Contributed by Nick @ Beeper (@Fizzadar). ([\#17032](element-hq/synapse#17032), [\#17096](element-hq/synapse#17096)) - Fix mypy with latest Twisted release. ([\#17036](element-hq/synapse#17036)) - Bump minimum supported Rust version to 1.66.0. ([\#17079](element-hq/synapse#17079)) - Add helpers to transform Twisted requests to Rust http Requests/Responses. ([\#17081](element-hq/synapse#17081)) - Fix type annotation for `visited_chains` after `mypy` upgrade. ([\#17125](element-hq/synapse#17125)) * Bump anyhow from 1.0.81 to 1.0.82. ([\#17095](element-hq/synapse#17095)) * Bump peaceiris/actions-gh-pages from 3.9.3 to 4.0.0. ([\#17087](element-hq/synapse#17087)) * Bump peaceiris/actions-mdbook from 1.2.0 to 2.0.0. ([\#17089](element-hq/synapse#17089)) * Bump pyasn1-modules from 0.3.0 to 0.4.0. ([\#17093](element-hq/synapse#17093)) * Bump pygithub from 2.2.0 to 2.3.0. ([\#17092](element-hq/synapse#17092)) * Bump ruff from 0.3.5 to 0.3.7. ([\#17094](element-hq/synapse#17094)) * Bump sigstore/cosign-installer from 3.4.0 to 3.5.0. ([\#17088](element-hq/synapse#17088)) * Bump twine from 4.0.2 to 5.0.0. ([\#17091](element-hq/synapse#17091)) * Bump types-pillow from 10.2.0.20240406 to 10.2.0.20240415. ([\#17090](element-hq/synapse#17090))

@clokep

# Synapse 1.109.0 (2024-06-18) - Add the ability to auto-accept invites on the behalf of users. See the [`auto_accept_invites`](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#auto-accept-invites) config option for details. ([\#17147](element-hq/synapse#17147)) - Add experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync/e2ee` endpoint for to-device messages and device encryption info. ([\#17167](element-hq/synapse#17167)) - Support [MSC3916](matrix-org/matrix-spec-proposals#3916) by adding unstable media endpoints to `/_matrix/client`. ([\#17213](element-hq/synapse#17213)) - Add logging to tasks managed by the task scheduler, showing CPU and database usage. ([\#17219](element-hq/synapse#17219)) # Synapse 1.108.0 (2024-05-28) - Add a feature that allows clients to query the configured federation whitelist. Disabled by default. ([\#16848](element-hq/synapse#16848), [\#17199](element-hq/synapse#17199)) - Add the ability to allow numeric user IDs with a specific prefix when in the CAS flow. Contributed by Aurélien Grimpard. ([\#17098](element-hq/synapse#17098)) Synapse 1.107.0 (2024-05-14) - Add preliminary support for [MSC3823: Account Suspension](matrix-org/matrix-spec-proposals#3823). ([\#17051](element-hq/synapse#17051)) - Declare support for [Matrix v1.10](https://matrix.org/blog/2024/03/22/matrix-v1.10-release/). Contributed by @clokep. ([\#17082](element-hq/synapse#17082)) - Add support for [MSC4115: membership metadata on events](matrix-org/matrix-spec-proposals#4115). ([\#17104](element-hq/synapse#17104), [\#17137](element-hq/synapse#17137)) # Synapse 1.106.0 (2024-04-30) - Send an email if the address is already bound to an user account. ([\#16819](element-hq/synapse#16819)) - Implement the rendezvous mechanism described by [MSC4108](matrix-org/matrix-spec-proposals#4108). ([\#17056](element-hq/synapse#17056)) - Support delegating the rendezvous mechanism described [MSC4108](matrix-org/matrix-spec-proposals#4108) to an external implementation. ([\#17086](element-hq/synapse#17086))

sandhose force-pushed the quenting/msc4108 branch from b7f14d6 to a887d43 Compare April 5, 2024 16:31

hughns mentioned this pull request Apr 10, 2024

Support for MSC4108 delegation #17070

Closed

sandhose force-pushed the quenting/msc4108 branch 3 times, most recently from c4b39e9 to 9c8091a Compare April 11, 2024 09:13

hughns requested changes Apr 11, 2024

View reviewed changes

hughns mentioned this pull request Apr 11, 2024

MSC4108: Mechanism to allow OIDC sign in and E2EE set up via QR code matrix-org/matrix-spec-proposals#4108

Open

12 tasks

sandhose force-pushed the quenting/msc4108 branch 3 times, most recently from 468c0a4 to 7c0c6c1 Compare April 12, 2024 12:20

hughns reviewed Apr 12, 2024

View reviewed changes

synapse/http/server.py Outdated Show resolved Hide resolved

sandhose force-pushed the quenting/msc4108 branch from 7c0c6c1 to ea51b4b Compare April 12, 2024 13:03

This was referenced Apr 12, 2024

Bump minimum required Rust version to 1.66.0 #17079

Merged

Helpers to transform Twisted requests to Rust http Requests/Responses #17081

Merged

sandhose force-pushed the quenting/msc4108 branch from ea51b4b to daf2985 Compare April 15, 2024 07:04

sandhose changed the base branch from develop to quenting/merge April 15, 2024 07:05

sandhose force-pushed the quenting/msc4108 branch from daf2985 to a87243f Compare April 15, 2024 07:19

sandhose force-pushed the quenting/merge branch from 68834f4 to 591807a Compare April 15, 2024 08:01

sandhose force-pushed the quenting/msc4108 branch from a87243f to 8ebe974 Compare April 15, 2024 08:01

sandhose mentioned this pull request Apr 15, 2024

Support for MSC4108 via delegation #17086

Merged

sandhose force-pushed the quenting/merge branch from 591807a to 272fa7e Compare April 15, 2024 08:20

sandhose force-pushed the quenting/msc4108 branch from 8ebe974 to d4dec65 Compare April 15, 2024 08:22

sandhose added 7 commits April 18, 2024 12:20

Setup the basic Python -> Rust call for the rendezvous endpoint

1697a80

WIP rendezvous implementation

bb86213

Some more rendezvous implementation

cd9809e

Return better errors

08b4525

Add config flag, rename to RendezVousHandler & other fixes

73d3152

Prepare for looping eviction calls

a34c8f7

Use Synapse's clock to generate timestamps

317528d

hughns and others added 7 commits April 18, 2024 12:20

Set rest of required CORS headers

4bf190f

Implement session eviction

60e0a48

Do not allow GETting/PUTting expired sessions

5f9fe56

Move the rendezvous session handler to the _synapse/ namespace

4157151

Add tests for the rendezvous implementation

37af4f4

Newsfile

bc5d2d0

Make the capacity, max payload size and eviction interval configurable

b04f043

This does not add the config bits, but does add the plumbing to set it from the Python size

sandhose changed the base branch from quenting/merge to develop April 18, 2024 10:25

sandhose force-pushed the quenting/msc4108 branch from 2db9345 to b04f043 Compare April 18, 2024 10:25

sandhose marked this pull request as ready for review April 18, 2024 10:31

sandhose requested a review from a team as a code owner April 18, 2024 10:31

hughns reviewed Apr 18, 2024

View reviewed changes

anoadragon453 requested changes Apr 19, 2024

View reviewed changes

rust/src/rendezvous/mod.rs Outdated Show resolved Hide resolved

rust/src/rendezvous/mod.rs Outdated Show resolved Hide resolved

rust/src/rendezvous/mod.rs Show resolved Hide resolved

hughns mentioned this pull request Apr 22, 2024

MSC4108 4KB maximum + 60 second TTL #17113

Merged

MSC4108 4KB maximum + 60 second TTL (#17113)

de01983

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

hughns mentioned this pull request Apr 25, 2024

Restrict MSC4108 content-type to text/plain #17122

Merged

Restrict MSC4108 content-type to text/plain (#17122)

86c7797

anoadragon453 mentioned this pull request Apr 25, 2024

Add rate-limiting to the Rendezvous experimental endpoint #17123

Open

Merge branch 'develop' into quenting/msc4108

80e174f

anoadragon453 approved these changes Apr 25, 2024

View reviewed changes

anoadragon453 enabled auto-merge (squash) April 25, 2024 11:40

Merge branch 'develop' into quenting/msc4108

5a63a46

anoadragon453 merged commit 2e92b71 into develop Apr 25, 2024
38 checks passed

anoadragon453 deleted the quenting/msc4108 branch April 25, 2024 12:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC4108 implementation #17056

MSC4108 implementation #17056

sandhose commented Apr 5, 2024 •

edited

Loading

CLAassistant commented Apr 10, 2024 •

edited

Loading

hughns Apr 11, 2024

sandhose Apr 18, 2024

hughns Apr 18, 2024

hughns Apr 18, 2024

anoadragon453 left a comment

anoadragon453 left a comment

anoadragon453 commented Apr 25, 2024 •

edited

Loading

MSC4108 implementation #17056

MSC4108 implementation #17056

Conversation

sandhose commented Apr 5, 2024 • edited Loading

CLAassistant commented Apr 10, 2024 • edited Loading

hughns Apr 11, 2024

Choose a reason for hiding this comment

sandhose Apr 18, 2024

Choose a reason for hiding this comment

hughns Apr 18, 2024

Choose a reason for hiding this comment

hughns Apr 18, 2024

Choose a reason for hiding this comment

anoadragon453 left a comment

Choose a reason for hiding this comment

anoadragon453 left a comment

Choose a reason for hiding this comment

anoadragon453 commented Apr 25, 2024 • edited Loading

sandhose commented Apr 5, 2024 •

edited

Loading

CLAassistant commented Apr 10, 2024 •

edited

Loading

anoadragon453 commented Apr 25, 2024 •

edited

Loading