MSC4143: MatrixRTC #4143
MSC4143: MatrixRTC #4143
Conversation
toger5
force-pushed
the
toger5/matrixRTC
branch
2 times, most recently
from
d717c0b to
cff8291
Compare
turt2live
added
voip
proposal
Signed-off-by: Timo K <toger5@hotmail.de>
proposals/4143-matrix-rtc.md
Outdated
| ## Unstable prefix | ||
|
|
||
| The state events and the well_known key introduced in this MSC use the unstable prefix | ||
| `org.matrix.msc4143.` instead of `m.` as used in the text. |
There was a problem hiding this comment.
empirically we seem to be using org.matrix.msc3401.call.member rather than org.matrix.msc4143.rtc.member in Element?
There was a problem hiding this comment.
The current implementation is still using the msc3401 prefix. That is wrong and will be addressed. There is still the open topic of how exactly we want to do the state keys and the event ownership and on top of that we have plans for how to index rtc member events in a better way.
The reason we changed it to (...call... ->) ...rtc... is, that we need the call namespace for the particular video call matrixRTC application (session type) of calling over MatrixRTC. Using that word for both. Matrix rtc sessions in general and calls will be confusing in the long run.
proposals/4143-matrix-rtc.md
Outdated
| `created_ts()`+`device_id`. This is why the `m.rtc.member` events deliberately do NOT include a `membership_id`. | ||
|
|
||
| Other then the membership sessions, there is **no event** to represent a rtc session (containing all members). | ||
| Such an event would include shared information, and deciding who has authority over that is not trivial. |
There was a problem hiding this comment.
I continue to trip over whether it is wise to force clients to read all possible m.rtc.member events to figure out if a call is happening, and who created it.
I /think/ that a better reason for not having an m.rtc state event describing the existence of an RTC session is that you'd have to handle disconnection semantics on it similar to delayed-events for m.rtc.member... at which point, why not leverage the membership events?
However, it still feels REALLY weird to not have something in state telling you whether semantically a call is intended to be happening now (and what that sort of call is, when it began, and who initiated it) - versus having to infer it.
There was a problem hiding this comment.
Alternatively, if the thought experiment is "what if two users both create an m.rtc state event on different forks of the DAG at the same time?" ... is that really so bad? and does aggregating m.rtc.member state actually make it better? if so, how?
In other words, we actually need to justify the lack of m.rtc state event much better here, imo. In particular, having somewhere to store the metadata about the call at the point of creation (its name, its ID, whether it's intended to be a voice/video room or a group call or a conference, etc)
There was a problem hiding this comment.
There is a large list of reasons and I 100% support the idea to go into detail in the MSC to justify this approach.
As for this comment I will just give a couple of short examples/arguments:
The security and "trolling" surface is huge. If we have one state event we either limit who can create a call or we allow everyone to mess with the event. This can go from smaller issues like ending the call for fun to larger issues like changing where the call is happening without the members noticing it. (in a LK world at least)
But even if everyone plays fair, a call can be stopped by a delayed event because the creator failed to send the refresh event. This now disconnects everyone from the call.
Independent how we make things behave, if we have a public/shared event controlling the experience for everyone we switch from:
- If there is a client with issues (or user that actively introduces issues) that user has a degraded experience
- If there is a client with issues (or user that actively introduces issues) everyone experience can be broken.
In the context of matrix where there is no central entity controlling the clients the seconds seems to be the only valid option.
|
|
||
| - [`m.call`](www.example.com) TODO: create `m.call` MSC and add link here. | ||
|
|
||
| ## Potential issues |
There was a problem hiding this comment.
The fact we don't reference how to tell users that a call is happening (i.e. m.call.notify) is very disorienting here.
proposals/4143-matrix-rtc.md
Outdated
| ``` | ||
|
|
||
| > [!NOTE] | ||
| > This relies on [MSC3757](https://github.com/matrix-org/matrix-spec-proposals/pull/3757). |
There was a problem hiding this comment.
MSC3757 should be added to the dependencies section at the bottom
proposals/4143-matrix-rtc.md
Outdated
| // event type: "m.rtc.member" | ||
| // event key: "@user:matrix.domain_DEVICEID" | ||
| { | ||
| "leave_reason": "CONNECTION_LOST" |
There was a problem hiding this comment.
this should be namespaced. what other leave reasons are there?
proposals/4143-matrix-rtc.md
Outdated
| fields are explained and how the communication with the possible foci is | ||
| defined: | ||
|
|
||
| - [`m.call`](www.example.com) TODO: create `m.call` MSC and add link here. |
There was a problem hiding this comment.
Something which is very unclear right now is the lifecycle of a call (and the draft of MSC4196 doesn't really help).
Specifically, what has happened to m.call.invite? m.call.answer? m.call.select_answer? m.call.hangup? The intention of MSC3401 was to keep using these (as per https://github.com/matrix-org/matrix-spec-proposals/blob/matthew/group-voip/proposals/3401-group-voip.md#basic-call) even when an SFU is involved, in order to preserve signalling for:
- Call was placed but isn't ringing yet (not that we have in legacy VoIP)
- Call is ringing
- Caller gave up and stopped ringing
- Call has been answered on a device
- Call was rejected locally
- Call was rejected on all devices
- Call was answered
- Call was hung up
i.e. all the state machine transitions that go into the lifecycle of a call.
I'm very worried that this seems to have been collapsed to a single m.call.notify event from MSC4075 + m.rtc.member events here, which seems to be missing most of the above. For instance, I can't see how the caller tells the callee that it's no longer ringing them - not to mention stuff like early media (MSC3635) or ringback tones.
Legacy VoIP was already a dumbed down version of SIP, and I'm worried that the lack of signalling semantics here is going to make mimicing SIP or PSTN semantics really hard (let alone bridging to it), and would be a backwards step from legacy voip.
There was a problem hiding this comment.
The reason that information about this is so thin here is that this proposal tries to define the line between matrixRTC and the special case matrix rtc use case of calls.
The focus should be to find the minimum requirements for a rtc session so clients can implement everything from this msc and they will be able to have a rough idea what is happening in a room.
Everything that is not applicable to all kinds of sessions should be part of their dedicated msc's.
Calling in particular has a very rich setup/notification cycle that is not necessarily part of every rtc session.
Th idea very much is however to fully support this list:
- Call was placed but isn't ringing yet (not that we have in legacy VoIP) !! This one is TBD
- Call is ringing
- Caller gave up and stopped ringing
- Call has been answered on a device
- Call was rejected locally
- Call was rejected on all devices
- Call was answered
- Call was hung up
This proposals tries to not enforce this for matrixRTC in general however.
Your comment makes me wonder if there is any benefit in reusing the already existing legacy calling events for m.call.invite m.call.answer m.call.select_answer m.call.hangup ...
There was a problem hiding this comment.
this proposal tries to define the line between matrixRTC and the special case matrix rtc use case of calls.
are we sure this is not an unnecessary abstraction right now? we could add it later, if/when we have a use case for MatrixRTC which doesn't involve calling? and meanwhile keep the MSCs easier to follow and less fragmented?
There was a problem hiding this comment.
Your comment makes me wonder if there is any benefit in reusing the already existing legacy calling events for m.call.invite m.call.answer m.call.select_answer m.call.hangup ...
There may not be any advantage to reusing those events, but it's very hard to tell when I can't figure out how those semantics map onto a MatrixRTC call life cycle today. My spidey sense is that having to infer the call state machine out of m.rtc.member and m.rtc.notify events is going to be fragile, hard to reason about, and hard to extend (e.g. how would you do early media, for compatibility with MSC3635?) - rather than throwing explicit events around when things happen. But am very happy to be corrected... especially if the rationale ends up in an MSC :)
Rendered
To-do:
Pull Request Checklist
#matrix-spec:matrix.org to
get feedback on this PR.