Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce MatrixRTCSession lower level group call primitive #3663

Merged
merged 89 commits into from
Sep 12, 2023
Merged
Show file tree
Hide file tree
Changes from 86 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
af10b0c
Add hacky option to disable the actual calling part of group calls.
dbkr Jun 14, 2023
4b3406d
Put LiveKit info into the `m.call` state event (#3522)
SimonBrandner Jul 5, 2023
f67ad33
Send 'contentLoaded' event
dbkr Jul 7, 2023
6e23dd7
Merge pull request #3556 from matrix-org/dbkr/sendcontentloaded
dbkr Jul 10, 2023
44cebce
Add comment on updating the livekit service URL
dbkr Jul 10, 2023
c746ab5
Merge pull request #3563 from matrix-org/dbkr/comment_lk_state_update
dbkr Jul 10, 2023
07dd1bd
Appease CI on `livekit` branch (#3566)
SimonBrandner Jul 10, 2023
71c944e
Update codeowners on `livekit` branch (#3567)
SimonBrandner Jul 10, 2023
2756ea0
add getOpenIdToken to embedded client backend
toger5 Jul 10, 2023
9eae6b2
add test and update comment
toger5 Jul 10, 2023
8297588
Merge `develop` into `livekit` (#3569)
SimonBrandner Jul 10, 2023
29b4074
Revert "Merge `develop` into `livekit`" (#3572)
SimonBrandner Jul 10, 2023
6fbd941
Merge branch 'develop' into lk_merge_main
toger5 Jul 10, 2023
9ba44b9
Merge pull request #3571 from matrix-org/toger5/oidcEmbedded
toger5 Jul 11, 2023
d79d9ae
Merge pull request #3573 from matrix-org/lk_merge_main
toger5 Jul 11, 2023
d4d73a6
Don't update calls with no livekit URL & expose method to update it i…
dbkr Jul 12, 2023
eb6e204
Fix other instances of passing focusInfo / livekit url
dbkr Jul 12, 2023
6139c93
Add temporary setter
dbkr Jul 12, 2023
b698217
Merge pull request #3598 from matrix-org/dbkr/update_lk_urls
dbkr Jul 12, 2023
11ce532
WIP refactor for removing m.call events
dbkr Aug 16, 2023
38f9b56
Always remember rtcsessions since we need to only have one instance
dbkr Aug 18, 2023
f5f63d6
Merge remote-tracking branch 'origin/develop' into dbkr/matrixrtcsession
dbkr Aug 18, 2023
426f498
Fix tests
dbkr Aug 18, 2023
542ce1f
Fix import loop
dbkr Aug 18, 2023
79ad566
Fix more cyclic imports & tests
dbkr Aug 18, 2023
3d715fc
Test session joining
dbkr Aug 18, 2023
ac6fece
Attempt to make tests happy
dbkr Aug 18, 2023
210c3bb
Always leave calls in the tests to clean up
dbkr Aug 18, 2023
bd10507
comment + desperate attempt to work out what's failing
dbkr Aug 18, 2023
b1dfdee
More test debugging
dbkr Aug 18, 2023
03868f8
Okay, so these ones are fine?
dbkr Aug 18, 2023
c9de1cb
Stop more timers and hopefully have happy tests
dbkr Aug 18, 2023
3de7b32
Test no rejoin
dbkr Aug 18, 2023
73dedc4
Test malformed m.call.member events
dbkr Aug 18, 2023
6aeeb7c
Test event emitting
dbkr Aug 21, 2023
989c4f0
Test getActiveFoci()
dbkr Aug 21, 2023
88d85b4
Test event emitting (and also fix it)
dbkr Aug 21, 2023
a07f93f
Test membership updating & pruning on join
dbkr Aug 22, 2023
86a25b5
Test getOldestMembership()
dbkr Aug 22, 2023
f0a37cb
Test member event renewal
dbkr Aug 22, 2023
0d46aeb
Merge remote-tracking branch 'origin/develop' into dbkr/matrixrtcsession
dbkr Aug 22, 2023
c8ae665
Don't start the rtc manager until the client has synced
dbkr Aug 23, 2023
770d16e
Fix type
dbkr Aug 23, 2023
5be2d7c
Remove listeners added in constructor
dbkr Aug 23, 2023
71ab476
Stop the client here too
dbkr Aug 23, 2023
2e1aaa8
Stop the client here also also
dbkr Aug 23, 2023
bcff039
ARGH. Disable tests to work out which one is causing the exception
dbkr Aug 23, 2023
2886455
Disable everything
dbkr Aug 23, 2023
b4c40ef
Re-jig to avoid setting listeners in the constructor
dbkr Aug 23, 2023
5b9051e
No need to rename this anymore
dbkr Aug 23, 2023
c271625
argh, remove the right listener
dbkr Aug 23, 2023
5e2a555
Is it this test???
dbkr Aug 23, 2023
591df95
Re-enable some tests
dbkr Aug 23, 2023
5145b44
Try mocking getRooms to return something valid
dbkr Aug 23, 2023
1c96fc8
Re-enable other tests
dbkr Aug 23, 2023
6811ba4
Give up trying to get the tests to work sensibly and deal with getRoo…
dbkr Aug 23, 2023
b18ae38
Oops, don't enable the ones that were skipped before
dbkr Aug 23, 2023
40fb4ab
One more try at the sensible way
dbkr Aug 23, 2023
50da896
Didn't work, go back to the hack way.
dbkr Aug 23, 2023
f612b76
Log when we manage to send the member event update
dbkr Aug 24, 2023
9cb1c20
Support `getOpenIdToken()` in embedded mode (#3676)
SimonBrandner Aug 25, 2023
2047c98
Call `sendContentLoaded()` (#3677)
SimonBrandner Aug 25, 2023
1a0718f
Start MatrixRTC in embedded mode (#3679)
SimonBrandner Aug 28, 2023
c444e37
Reschedule the membership event check
dbkr Aug 29, 2023
edc977b
Bump widget api version
dbkr Aug 29, 2023
e690b71
Add mock for sendContentLoaded()
dbkr Aug 29, 2023
f2ce658
Merge branch 'develop' into dbkr/matrixrtcsession
dbkr Aug 30, 2023
4ea0754
More log detail
dbkr Aug 31, 2023
cb3f9ea
Fix tests
dbkr Aug 31, 2023
eb25a28
Simplify updateCallMembershipEvent a bit
dbkr Aug 31, 2023
861b1e9
Split up updateCallMembershipEvent some more
dbkr Aug 31, 2023
fc2671d
Merge remote-tracking branch 'origin/develop' into dbkr/matrixrtcsession
dbkr Aug 31, 2023
395b38d
Typo
dbkr Sep 6, 2023
7a024f2
Expand comment
dbkr Sep 6, 2023
0195eb2
Add comment
dbkr Sep 6, 2023
0f4e90b
More comments
dbkr Sep 6, 2023
59bbd17
Better comment
dbkr Sep 6, 2023
f2301ee
Sesson
dbkr Sep 6, 2023
1edb13f
Rename some variables
dbkr Sep 6, 2023
382d12d
Comment
dbkr Sep 6, 2023
e610f36
Merge remote-tracking branch 'origin/develop' into dbkr/matrixrtcsession
dbkr Sep 6, 2023
e7109c8
Remove unused method
dbkr Sep 7, 2023
268a147
Wrap updatecallMembershipEvent so it only runs one at a time
dbkr Sep 8, 2023
19bf62a
Do another update if another one is triggered while the update happens
dbkr Sep 8, 2023
d569ff7
Merge remote-tracking branch 'origin/livekit' into dbkr/matrixrtcsession
dbkr Sep 11, 2023
cc98caa
Make triggerCallMembershipEventUpdate async
dbkr Sep 11, 2023
767a468
Fix test & some missed timer removals
dbkr Sep 11, 2023
21fb752
Mark session manager as unstable
dbkr Sep 12, 2023
66eaebf
Merge branch 'develop' into dbkr/matrixrtcsession
dbkr Sep 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,6 +1 @@
* @matrix-org/element-web
/.github/workflows/** @matrix-org/element-web-app-team
/package.json @matrix-org/element-web-app-team
/yarn.lock @matrix-org/element-web-app-team
/src/webrtc @matrix-org/element-call-reviewers
/spec/*/webrtc @matrix-org/element-call-reviewers
* @matrix-org/element-call-reviewers
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
"jwt-decode": "^3.1.2",
"loglevel": "^1.7.1",
"matrix-events-sdk": "0.0.1",
"matrix-widget-api": "^1.5.0",
"matrix-widget-api": "^1.6.0",
"oidc-client-ts": "^2.2.4",
"p-retry": "4",
"sdp-transform": "^2.14.1",
Expand Down
1 change: 1 addition & 0 deletions spec/test-utils/webrtc.ts
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,7 @@ export class MockCallMatrixClient extends TypedEventEmitter<EmittedEvents, Emitt

public getRooms = jest.fn<Room[], []>().mockReturnValue([]);
public getRoom = jest.fn();
public getFoci = jest.fn();

public supportsThreads(): boolean {
return true;
Expand Down
29 changes: 27 additions & 2 deletions spec/unit/embedded.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,14 @@ limitations under the License.
// eslint-disable-next-line no-restricted-imports
import { EventEmitter } from "events";
import { MockedObject } from "jest-mock";
import { WidgetApi, WidgetApiToWidgetAction, MatrixCapabilities, ITurnServer, IRoomEvent } from "matrix-widget-api";
import {
WidgetApi,
WidgetApiToWidgetAction,
MatrixCapabilities,
ITurnServer,
IRoomEvent,
IOpenIDCredentials,
} from "matrix-widget-api";

import { createRoomWidgetClient, MsgType } from "../../src/matrix";
import { MatrixClient, ClientEvent, ITurnServer as IClientTurnServer } from "../../src/client";
Expand All @@ -33,6 +40,12 @@ import { MatrixEvent } from "../../src/models/event";
import { ToDeviceBatch } from "../../src/models/ToDeviceMessage";
import { DeviceInfo } from "../../src/crypto/deviceinfo";

const testOIDCToken = {
access_token: "12345678",
expires_in: "10",
matrix_server_name: "homeserver.oabc",
token_type: "Bearer",
};
class MockWidgetApi extends EventEmitter {
public start = jest.fn();
public requestCapability = jest.fn();
Expand All @@ -49,8 +62,15 @@ class MockWidgetApi extends EventEmitter {
public sendRoomEvent = jest.fn(() => ({ event_id: `$${Math.random()}` }));
public sendStateEvent = jest.fn();
public sendToDevice = jest.fn();
public requestOpenIDConnectToken = jest.fn(() => {
return testOIDCToken;
return new Promise<IOpenIDCredentials>(() => {
return testOIDCToken;
});
});
public readStateEvents = jest.fn(() => []);
public getTurnServers = jest.fn(() => []);
public sendContentLoaded = jest.fn();

public transport = { reply: jest.fn() };
}
Expand Down Expand Up @@ -285,7 +305,12 @@ describe("RoomWidgetClient", () => {
expect(await emittedSync).toEqual(SyncState.Syncing);
});
});

describe("oidc token", () => {
it("requests an oidc token", async () => {
await makeClient({});
expect(await client.getOpenIdToken()).toStrictEqual(testOIDCToken);
});
});
it("gets TURN servers", async () => {
const server1: ITurnServer = {
uris: [
Expand Down
139 changes: 139 additions & 0 deletions spec/unit/matrixrtc/CallMembership.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

import { MatrixEvent } from "../../../src";
import { CallMembership, CallMembershipData } from "../../../src/matrixrtc/CallMembership";

const membershipTemplate: CallMembershipData = {
call_id: "",
scope: "m.room",
application: "m.call",
device_id: "AAAAAAA",
expires: 5000,
};

function makeMockEvent(originTs = 0): MatrixEvent {
return {
getTs: jest.fn().mockReturnValue(originTs),
sender: {
userId: "@alice:example.org",
},
} as unknown as MatrixEvent;
}

describe("CallMembership", () => {
it("rejects membership with no expiry", () => {
expect(() => {
new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { expires: undefined }));
}).toThrow();
});

it("rejects membership with no device_id", () => {
expect(() => {
new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { device_id: undefined }));
}).toThrow();
});

it("rejects membership with no call_id", () => {
expect(() => {
new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { call_id: undefined }));
}).toThrow();
});

it("rejects membership with no scope", () => {
expect(() => {
new CallMembership(makeMockEvent(), Object.assign({}, membershipTemplate, { scope: undefined }));
}).toThrow();
});

it("uses event timestamp if no created_ts", () => {
const membership = new CallMembership(makeMockEvent(12345), membershipTemplate);
expect(membership.createdTs()).toEqual(12345);
});

it("uses created_ts if present", () => {
const membership = new CallMembership(
makeMockEvent(12345),
Object.assign({}, membershipTemplate, { created_ts: 67890 }),
);
expect(membership.createdTs()).toEqual(67890);
});

it("computes absolute expiry time", () => {
const membership = new CallMembership(makeMockEvent(1000), membershipTemplate);
expect(membership.getAbsoluteExpiry()).toEqual(5000 + 1000);
});

it("considers memberships unexpired if local age low enough", () => {
const fakeEvent = makeMockEvent(1000);
fakeEvent.getLocalAge = jest.fn().mockReturnValue(3000);
const membership = new CallMembership(fakeEvent, membershipTemplate);
expect(membership.isExpired()).toEqual(false);
});

it("considers memberships expired when local age large", () => {
const fakeEvent = makeMockEvent(1000);
fakeEvent.getLocalAge = jest.fn().mockReturnValue(6000);
const membership = new CallMembership(fakeEvent, membershipTemplate);
expect(membership.isExpired()).toEqual(true);
});

it("returns active foci", () => {
const fakeEvent = makeMockEvent();
const mockFocus = { type: "this_is_a_mock_focus" };
const membership = new CallMembership(
fakeEvent,
Object.assign({}, membershipTemplate, { foci_active: [mockFocus] }),
);
expect(membership.getActiveFoci()).toEqual([mockFocus]);
});

describe("expiry calculation", () => {
let fakeEvent: MatrixEvent;
let membership: CallMembership;

beforeEach(() => {
// server origin timestamp for this event is 1000
fakeEvent = makeMockEvent(1000);
// our clock would have been at 2000 at the creation time (our clock at event receive time - age)
// (ie. the local clock is 1 second ahead of the servers' clocks)
fakeEvent.localTimestamp = 2000;

// for simplicity's sake, we say that the event's age is zero
fakeEvent.getLocalAge = jest.fn().mockReturnValue(0);

membership = new CallMembership(fakeEvent!, membershipTemplate);

jest.useFakeTimers();
});

afterEach(() => {
jest.useRealTimers();
});

it("converts expiry time into local clock", () => {
// for sanity's sake, make sure the server-relative expiry time is what we expect
expect(membership.getAbsoluteExpiry()).toEqual(6000);
// therefore the expiry time converted to our clock should be 1 second later
expect(membership.getLocalExpiry()).toEqual(7000);
});

it("calculates time until expiry", () => {
jest.setSystemTime(2000);
expect(membership.getMsUntilExpiry()).toEqual(5000);
});
});
});
Loading
Loading