Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry to-device messages #2549

Merged
merged 11 commits into from
Aug 3, 2022
Merged

Retry to-device messages #2549

merged 11 commits into from
Aug 3, 2022

Conversation

dbkr
Copy link
Member

@dbkr dbkr commented Jul 28, 2022

This adds a queueToDevice API alongside sendToDevice which is a
much higher-level API that adds the messages to a queue, stored in
persistent storage, and retries them periodically. Also converts
sending of megolm keys to use the new API.

Other uses of sendToDevice are nopt converted in this PR, but could
be later.

Increments the version of the IndexedDB sync store, so a logout/login will be required if downgrading back to an earlier version after running this version.

Requires matrix-org/matrix-mock-request#17
Fixes element-hq/element-web#12851

Checklist

  • Tests written for new code (and old code if feasible)
  • Linter and other CI checks pass
  • Sign-off given on the changes (see CONTRIBUTING.md)

Here's what your changelog entry will look like:

✨ Features

dbkr added 2 commits July 28, 2022 16:07
This adds a queueToDevice API alongside sendToDevice which is a
much higher-level API that adds the messages to a queue, stored in
persistent storage, and retries them periodically. Also converts
sending of megolm keys to use the new API.

Other uses of sendToDevice are nopt converted in this PR, but could
be later.

Requires matrix-org/matrix-mock-request#17
@dbkr dbkr requested a review from a team as a code owner July 28, 2022 15:14
dbkr added a commit that referenced this pull request Jul 29, 2022
...onto group call branch

This commit just has the merge with encryptAndSendKeysToDevices
converted, but not other calls yet.
Copy link
Contributor

@germain-gg germain-gg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise lgtm

for (let i = 0; i < batch.batch.length; i += MAX_BATCH_SIZE) {
batches.push({
eventType: batch.eventType,
batch: batch.batch.slice(i, i + 20),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the 20 here the same as MAX_BATCH_SIZE ? or is it different?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, it was - that was a weird brainfart, thanks for catching that!

src/ToDeviceMessageQueue.ts Outdated Show resolved Hide resolved
src/store/memory.ts Outdated Show resolved Hide resolved
src/store/stub.ts Outdated Show resolved Hide resolved
src/store/indexeddb-local-backend.ts Outdated Show resolved Hide resolved
dbkr and others added 5 commits July 29, 2022 15:31
Co-authored-by: Germain <germains@element.io>
Co-authored-by: Germain <germains@element.io>
Co-authored-by: Germain <germains@element.io>
@dbkr dbkr merged commit 7e784da into develop Aug 3, 2022
@dbkr dbkr deleted the dbkr/queueToDevice branch August 3, 2022 12:32
dbkr added a commit that referenced this pull request Aug 4, 2022
* Prepare changelog for v19.2.0-rc.1

* v19.2.0-rc.1

* Sliding sync: add missing filters from latest MSC

* Gracefully handle missing room_ids

* Prepare changelog for v19.2.0

* v19.2.0

* Resetting package fields for development

* Use EventType enum values instead of hardcoded strings (#2557)

* Retry to-device messages (#2549)

* Retry to-device messages

This adds a queueToDevice API alongside sendToDevice which is a
much higher-level API that adds the messages to a queue, stored in
persistent storage, and retries them periodically. Also converts
sending of megolm keys to use the new API.

Other uses of sendToDevice are nopt converted in this PR, but could
be later.

Requires matrix-org/matrix-mock-request#17

* Bump matrix-mock-request

* Add more waits to make indexeddb tests pass

* Switch some test expectations to queueToDevice

* Stop straight away if the client has been stopped

Hopefully will fix tests being flakey and logging after tests have
finished.

* Add return types & fix constant usage

* Fix return type

Co-authored-by: Germain <germains@element.io>

* Fix return type

Co-authored-by: Germain <germains@element.io>

* Fix return type

Co-authored-by: Germain <germains@element.io>

* Stop the client in all test cases

Co-authored-by: Germain <germains@element.io>

* Add support for sending user-defined encrypted to-device messages (#2528)

* Add support for sending user-defined encrypted to-device messages

This is a port of the same change from the robertlong/group-call branch.

* Fix tests

* Expose the method in MatrixClient

* Fix a code smell

* Fix types

* Test the MatrixClient method

* Fix some types in Crypto test suite

* Test the Crypto method

* Fix tests

* Upgrade matrix-mock-request

* Move useRealTimers to afterEach

* Remove stream-replacement (#2551)

* Reintroduce setNewStream method, fix test, update yarn.lock

Co-authored-by: RiotRobot <releases@riot.im>
Co-authored-by: Kegan Dougal <kegan@matrix.org>
Co-authored-by: Germain <germains@element.io>
Co-authored-by: Robin <robin@robin.town>
Co-authored-by: Šimon Brandner <simon.bra.ag@gmail.com>
su-ex added a commit to SchildiChat/matrix-js-sdk that referenced this pull request Aug 22, 2022
* Add txn_id support to sliding sync ([\matrix-org#2567](matrix-org#2567)).
* Emit an event when the client receives TURN servers ([\matrix-org#2529](matrix-org#2529)).
* Add support for stable prefixes for MSC2285 ([\matrix-org#2524](matrix-org#2524)).
* Remove stream-replacement ([\matrix-org#2551](matrix-org#2551)).
* Add support for sending user-defined encrypted to-device messages ([\matrix-org#2528](matrix-org#2528)).
* Retry to-device messages ([\matrix-org#2549](matrix-org#2549)). Fixes element-hq/element-web#12851.
* Sliding sync: add missing filters from latest MSC ([\matrix-org#2555](matrix-org#2555)).
* Use stable prefixes for MSC3827 ([\matrix-org#2537](matrix-org#2537)).
* Fix: Handle parsing of a beacon info event without asset ([\matrix-org#2591](matrix-org#2591)). Fixes element-hq/element-web#23078.
* Fix finding event read up to if stable private read receipts is missing ([\matrix-org#2585](matrix-org#2585)). Fixes element-hq/element-web#23027.
* Fixed a sliding sync issue where history could be interpreted as live events. ([\matrix-org#2583](matrix-org#2583)).
* Don't load the sync accumulator if there's already a sync persist in flight ([\matrix-org#2569](matrix-org#2569)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

We don't retry E2EE to-device messages at all.
2 participants