Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Enable cancellation for POST /_matrix/client/v3/keys/query #12890

Closed
1 of 4 tasks
squahtx opened this issue May 26, 2022 · 3 comments · Fixed by #13680
Closed
1 of 4 tasks

Enable cancellation for POST /_matrix/client/v3/keys/query #12890

squahtx opened this issue May 26, 2022 · 3 comments · Fixed by #13680
Assignees
Labels
S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.

Comments

@squahtx
Copy link
Contributor

squahtx commented May 26, 2022

POST /_matrix/client/v3/keys/query can take a long time.

When clients retry the request, the new request gets queued behind the previous one by E2eKeysHandler._query_devices_linearizer, so retrying a request that timed out only makes response times worse.

...
2022-05-26 14:34:57,053 - synapse.access.http.11104 - 450 - INFO - POST-2875183 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 447.546sec/-267.564sec (0.213sec, 0.015sec) (0.015sec/0.052sec/34) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:35:04,581 - synapse.access.http.11104 - 450 - INFO - POST-2875478 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 435.215sec/-255.214sec (0.224sec, 0.029sec) (0.023sec/0.064sec/40) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:35:04,591 - synapse.access.http.11104 - 450 - INFO - POST-2874965 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 467.875sec/-287.874sec (0.213sec, 0.012sec) (0.044sec/0.106sec/46) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:35:05,443 - synapse.access.http.11104 - 450 - INFO - POST-2875779 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 414.790sec/-234.788sec (0.207sec, 0.013sec) (0.017sec/0.062sec/35) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:35:12,246 - synapse.access.http.11104 - 450 - INFO - POST-2875953 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 409.941sec/-229.943sec (0.234sec, 0.016sec) (0.010sec/0.054sec/18) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:35:34,240 - synapse.access.http.11104 - 450 - INFO - POST-2875839 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 439.033sec/-259.028sec (0.253sec, 0.012sec) (0.017sec/0.042sec/35) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:35:42,721 - synapse.access.http.11104 - 450 - INFO - POST-2875818 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 448.897sec/-268.734sec (0.238sec, 0.015sec) (0.019sec/0.052sec/30) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:36:04,578 - synapse.access.http.11104 - 450 - INFO - POST-2876136 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 452.018sec/-272.030sec (0.187sec, 0.015sec) (0.013sec/0.046sec/26) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:36:04,957 - synapse.access.http.11104 - 450 - INFO - POST-2876300 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 442.008sec/-262.071sec (0.201sec, 0.012sec) (0.013sec/0.045sec/31) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
2022-05-26 14:36:11,865 - synapse.access.http.11104 - 450 - INFO - POST-2877424 - xxx.xxx.xxx.xxx - 11104 - {@xxxxxxxxx:matrix.org} Processed request: 385.515sec/-205.517sec (0.182sec, 0.014sec) (0.108sec/0.087sec/26) 0B 200! "POST /_matrix/client/v3/keys/query HTTP/1.1" "xxxxxxxxx" [0 dbevts]
...

To prevent the linearizer queue from building up too much, we can enable cancellation for the endpoint.

@squahtx squahtx added T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. S-Tolerable Minor significance, cosmetic issues, low or no impact to users. labels May 26, 2022
@ritikBhandari
Copy link

Hi,
I'd like to work on this issue.

@richvdh
Copy link
Member

richvdh commented Aug 12, 2022

Per #13518: this manifests in thousands of requests to /_synapse/replication/user_device_resync on the main process, which are handled by ReplicationUserDevicesResyncRestServlet.

@erikjohnston erikjohnston added S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. and removed S-Tolerable Minor significance, cosmetic issues, low or no impact to users. T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. labels Aug 18, 2022
@reivilibre
Copy link
Contributor

reivilibre commented Aug 30, 2022

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants