Restore async concurrency safety to websocket compressor #7865

bdraco · 2023-11-22T03:24:04Z

What do these changes do?

Restore async concurrency safety to websocket compressor

Are there changes in behavior for the user?

Related issue number

fixes #7859

Checklist

I think the code is well written
Unit tests for the changes exist
Documentation reflects the changes
If you provide code modification, please add yourself to CONTRIBUTORS.txt
- The format is <Name> <Surname>.
- Please keep alphabetical order, the file is sorted by names.
Add a new news fragment into the CHANGES folder
- name it <issue_id>.<type> for example (588.bugfix)
- if you don't have an issue_id change it to the pr id after creating the pr
- ensure type is one of the following:
 - .feature: Signifying a new feature.
 - .bugfix: Signifying a bug fix.
 - .doc: Signifying a documentation improvement.
 - .removal: Signifying a deprecation or removal of public API.
 - .misc: A ticket has been closed, but it is not of interest to users.
- Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

codecov · 2023-11-22T03:46:21Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (4d9fc63) 97.40% compared to head (6d89a7f) 97.40%.
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #7865   +/-   ##
=======================================
  Coverage   97.40%   97.40%           
=======================================
  Files         107      107           
  Lines       32207    32237   +30     
  Branches     3743     3749    +6     
=======================================
+ Hits        31371    31401   +30     
  Misses        632      632           
  Partials      204      204

Flag	Coverage Δ
CI-GHA	`97.32% <100.00%> (+<0.01%)`	⬆️
OS-Linux	`96.99% <100.00%> (+<0.01%)`	⬆️
OS-Windows	`95.48% <100.00%> (+<0.01%)`	⬆️
OS-macOS	`96.80% <100.00%> (-0.01%)`	⬇️
Py-3.10.11	`95.41% <100.00%> (+<0.01%)`	⬆️
Py-3.10.13	`96.79% <100.00%> (+<0.01%)`	⬆️
Py-3.11.6	`96.53% <100.00%> (+<0.01%)`	⬆️
Py-3.12.0	`96.60% <100.00%> (+<0.01%)`	⬆️
Py-3.8.10	`95.37% <100.00%> (+<0.01%)`	⬆️
Py-3.8.18	`96.72% <100.00%> (+<0.01%)`	⬆️
Py-3.9.13	`95.37% <100.00%> (+<0.01%)`	⬆️
Py-3.9.18	`96.75% <100.00%> (-0.01%)`	⬇️
Py-pypy7.3.13	`96.22% <100.00%> (+<0.01%)`	⬆️
VM-macos	`96.80% <100.00%> (-0.01%)`	⬇️
VM-ubuntu	`96.99% <100.00%> (+<0.01%)`	⬆️
VM-windows	`95.48% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

bdraco · 2023-11-22T07:46:49Z

For testing this I guess we gather a few writers to happen at the same time and than check the received messages are not corrupted

tests/test_websocket_writer.py

bdraco · 2023-11-22T10:30:42Z

I think this will solve it. I need to do more functional testing but my flight is taking off now

bdraco · 2023-11-22T11:44:47Z

functional testing looks good

doing performance testing now

bdraco · 2023-11-22T11:46:35Z

performance profile remains similar

tests/test_websocket_writer.py

Co-authored-by: Sam Bull <git@sambull.org>

…em up

aiohttp/http_websocket.py

Dreamsorcerer · 2023-11-22T19:48:53Z

@webknjaz I'd like to nominate @bdraco for membership to @aio-libs/aiohttp-dev. There's plenty of examples in the previous weeks of good quality PRs, with all the tests we'd expect: https://github.com/aio-libs/aiohttp/pulls?q=is%3Apr+author%3Abdraco+is%3Aclosed

Dreamsorcerer · 2023-11-23T18:34:05Z

I'd like to nominate @bdraco for membership

OK, done this properly now: #7880

aiohttp/http_websocket.py

patchback · 2023-11-24T14:15:17Z

Backport to 3.9: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 86a2396 on top of patchback/backports/3.9/86a23961531103ccc34853f67321c7d0f63797f5/pr-7865

Backporting merged PR #7865 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.9/86a23961531103ccc34853f67321c7d0f63797f5/pr-7865 upstream/3.9

Now, cherry-pick PR Restore async concurrency safety to websocket compressor #7865 contents into that branch:
```
$ git cherry-pick -x 86a23961531103ccc34853f67321c7d0f63797f5
```
If it'll yell at you with something like fatal: Commit 86a23961531103ccc34853f67321c7d0f63797f5 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x 86a23961531103ccc34853f67321c7d0f63797f5
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Restore async concurrency safety to websocket compressor #7865 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.9/86a23961531103ccc34853f67321c7d0f63797f5/pr-7865

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2023-11-24T14:15:23Z

Backport to 3.10: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 86a2396 on top of patchback/backports/3.10/86a23961531103ccc34853f67321c7d0f63797f5/pr-7865

Backporting merged PR #7865 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.10/86a23961531103ccc34853f67321c7d0f63797f5/pr-7865 upstream/3.10

Now, cherry-pick PR Restore async concurrency safety to websocket compressor #7865 contents into that branch:
```
$ git cherry-pick -x 86a23961531103ccc34853f67321c7d0f63797f5
```
If it'll yell at you with something like fatal: Commit 86a23961531103ccc34853f67321c7d0f63797f5 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x 86a23961531103ccc34853f67321c7d0f63797f5
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Restore async concurrency safety to websocket compressor #7865 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.10/86a23961531103ccc34853f67321c7d0f63797f5/pr-7865

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Fixes aio-libs#7859 (cherry picked from commit 86a2396)

Fixes #7859 (cherry picked from commit 86a2396)

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.9.0 to 3.9.1. <details> <summary>Release notes</summary> Sourced from <a href="https://github.com/aio-libs/aiohttp/releases">aiohttp's releases</a>. <blockquote> <h2>3.9.1</h2> <h2>Bugfixes</h2> <ul> <li> Fixed importing aiohttp under PyPy on Windows. (<a href="https://github.com/aio-libs/aiohttp/issues/7848">#7848</a>) </li> <li> Fixed async concurrency safety in websocket compressor. (<a href="https://github.com/aio-libs/aiohttp/issues/7865">#7865</a>) </li> <li> Fixed <code>ClientResponse.close()</code> releasing the connection instead of closing. (<a href="https://github.com/aio-libs/aiohttp/issues/7869">#7869</a>) </li> <li> Fixed a regression where connection may get closed during upgrade. -- by :user:<code>Dreamsorcerer</code> (<a href="https://github.com/aio-libs/aiohttp/issues/7879">#7879</a>) </li> <li> Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:<code>Dreamsorcerer</code> (<a href="https://github.com/aio-libs/aiohttp/issues/7895">#7895</a>) </li> </ul> <hr /> </blockquote> </details> <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst">aiohttp's changelog</a>. <blockquote> <h1>3.9.1 (2023-11-26)</h1> <h2>Bugfixes</h2> <ul> <li> Fixed importing aiohttp under PyPy on Windows. <code>[#7848](aio-libs/aiohttp#7848) <https://github.com/aio-libs/aiohttp/issues/7848></code>_ </li> <li> Fixed async concurrency safety in websocket compressor. <code>[#7865](aio-libs/aiohttp#7865) <https://github.com/aio-libs/aiohttp/issues/7865></code>_ </li> <li> Fixed <code>ClientResponse.close()</code> releasing the connection instead of closing. <code>[#7869](aio-libs/aiohttp#7869) <https://github.com/aio-libs/aiohttp/issues/7869></code>_ </li> <li> Fixed a regression where connection may get closed during upgrade. -- by :user:<code>Dreamsorcerer</code> <code>[#7879](aio-libs/aiohttp#7879) <https://github.com/aio-libs/aiohttp/issues/7879></code>_ </li> <li> Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:<code>Dreamsorcerer</code> <code>[#7895](aio-libs/aiohttp#7895) <https://github.com/aio-libs/aiohttp/issues/7895></code>_ </li> </ul> <hr /> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aio-libs/aiohttp/commit/6333c026422c6b0fe57ff63cde4104e2d00f47f4"><code>6333c02</code></a> Release v3.9.1 (<a href="https://github.com/aio-libs/aiohttp/issues/7911">#7911</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/9dbd273093d6af6f5e1481816b05a7192860b440"><code>9dbd273</code></a> [PR <a href="https://github.com/aio-libs/aiohttp/issues/7673">#7673</a>/aa7d1a8f backport][3.9] Document release process (<a href="https://github.com/aio-libs/aiohttp/issues/7909">#7909</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/dd175b6b89564dc74fba0692a8a5f9a9b38e528a"><code>dd175b6</code></a> Fix regression with connection upgrade (<a href="https://github.com/aio-libs/aiohttp/issues/7879">#7879</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7908">#7908</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/946523d6380bd79e13146557432f46f6f9bbd53f"><code>946523d</code></a> Fix flaky websocket test (<a href="https://github.com/aio-libs/aiohttp/issues/7902">#7902</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7904">#7904</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/ddc2a26c9e0c43fd1229e4424f2a30d1b10ced13"><code>ddc2a26</code></a> [PR <a href="https://github.com/aio-libs/aiohttp/issues/7896">#7896</a>/9a7cfe77 backport][3.9] Fix some flaky tests (<a href="https://github.com/aio-libs/aiohttp/issues/7900">#7900</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/2ae4d6ffdd015f622bfb75dee98ad629240cccc4"><code>2ae4d6f</code></a> Message is not upgraded if Upgrade header is missing (<a href="https://github.com/aio-libs/aiohttp/issues/7895">#7895</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7898">#7898</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/bb111012706d3ef9edc525be3d8d4df410ad847f"><code>bb11101</code></a> Restore async concurrency safety to websocket compressor (<a href="https://github.com/aio-libs/aiohttp/issues/7865">#7865</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7889">#7889</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/6dd0122417f00ed4b2b353226a1b164b6463a245"><code>6dd0122</code></a> Update dependabot.yml (<a href="https://github.com/aio-libs/aiohttp/issues/7888">#7888</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/41a9f1f5b9b2630b1f4b971811c7ef8f016262fb"><code>41a9f1f</code></a> Bump mypy from 1.7.0 to 1.7.1 (<a href="https://github.com/aio-libs/aiohttp/issues/7882">#7882</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/a04970150c6ce9fda22c9f63d947845f79148b4c"><code>a049701</code></a> Fix usage of proxy.py in test_proxy_functional (<a href="https://github.com/aio-libs/aiohttp/issues/7773">#7773</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7876">#7876</a>)</li> <li>Additional commits viewable in <a href="https://github.com/aio-libs/aiohttp/compare/v3.9.0...v3.9.1">compare view</a></li> </ul> </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aiohttp&package-manager=pip&previous-version=3.9.0&new-version=3.9.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

Fixes aio-libs#7859

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.9.0 to 3.9.1. <details> <summary>Release notes</summary> Sourced from <a href="https://github.com/aio-libs/aiohttp/releases">aiohttp's releases</a>. <blockquote> <h2>3.9.1</h2> <h2>Bugfixes</h2> <ul> <li> Fixed importing aiohttp under PyPy on Windows. (<a href="https://github.com/aio-libs/aiohttp/issues/7848">#7848</a>) </li> <li> Fixed async concurrency safety in websocket compressor. (<a href="https://github.com/aio-libs/aiohttp/issues/7865">#7865</a>) </li> <li> Fixed <code>ClientResponse.close()</code> releasing the connection instead of closing. (<a href="https://github.com/aio-libs/aiohttp/issues/7869">#7869</a>) </li> <li> Fixed a regression where connection may get closed during upgrade. -- by :user:<code>Dreamsorcerer</code> (<a href="https://github.com/aio-libs/aiohttp/issues/7879">#7879</a>) </li> <li> Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:<code>Dreamsorcerer</code> (<a href="https://github.com/aio-libs/aiohttp/issues/7895">#7895</a>) </li> </ul> <hr /> </blockquote> </details> <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst">aiohttp's changelog</a>. <blockquote> <h1>3.9.1 (2023-11-26)</h1> <h2>Bugfixes</h2> <ul> <li> Fixed importing aiohttp under PyPy on Windows. <code>[#7848](aio-libs/aiohttp#7848) <https://github.com/aio-libs/aiohttp/issues/7848></code>_ </li> <li> Fixed async concurrency safety in websocket compressor. <code>[#7865](aio-libs/aiohttp#7865) <https://github.com/aio-libs/aiohttp/issues/7865></code>_ </li> <li> Fixed <code>ClientResponse.close()</code> releasing the connection instead of closing. <code>[#7869](aio-libs/aiohttp#7869) <https://github.com/aio-libs/aiohttp/issues/7869></code>_ </li> <li> Fixed a regression where connection may get closed during upgrade. -- by :user:<code>Dreamsorcerer</code> <code>[#7879](aio-libs/aiohttp#7879) <https://github.com/aio-libs/aiohttp/issues/7879></code>_ </li> <li> Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:<code>Dreamsorcerer</code> <code>[#7895](aio-libs/aiohttp#7895) <https://github.com/aio-libs/aiohttp/issues/7895></code>_ </li> </ul> <hr /> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aio-libs/aiohttp/commit/6333c026422c6b0fe57ff63cde4104e2d00f47f4"><code>6333c02</code></a> Release v3.9.1 (<a href="https://github.com/aio-libs/aiohttp/issues/7911">#7911</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/9dbd273093d6af6f5e1481816b05a7192860b440"><code>9dbd273</code></a> [PR <a href="https://github.com/aio-libs/aiohttp/issues/7673">#7673</a>/aa7d1a8f backport][3.9] Document release process (<a href="https://github.com/aio-libs/aiohttp/issues/7909">#7909</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/dd175b6b89564dc74fba0692a8a5f9a9b38e528a"><code>dd175b6</code></a> Fix regression with connection upgrade (<a href="https://github.com/aio-libs/aiohttp/issues/7879">#7879</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7908">#7908</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/946523d6380bd79e13146557432f46f6f9bbd53f"><code>946523d</code></a> Fix flaky websocket test (<a href="https://github.com/aio-libs/aiohttp/issues/7902">#7902</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7904">#7904</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/ddc2a26c9e0c43fd1229e4424f2a30d1b10ced13"><code>ddc2a26</code></a> [PR <a href="https://github.com/aio-libs/aiohttp/issues/7896">#7896</a>/9a7cfe77 backport][3.9] Fix some flaky tests (<a href="https://github.com/aio-libs/aiohttp/issues/7900">#7900</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/2ae4d6ffdd015f622bfb75dee98ad629240cccc4"><code>2ae4d6f</code></a> Message is not upgraded if Upgrade header is missing (<a href="https://github.com/aio-libs/aiohttp/issues/7895">#7895</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7898">#7898</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/bb111012706d3ef9edc525be3d8d4df410ad847f"><code>bb11101</code></a> Restore async concurrency safety to websocket compressor (<a href="https://github.com/aio-libs/aiohttp/issues/7865">#7865</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7889">#7889</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/6dd0122417f00ed4b2b353226a1b164b6463a245"><code>6dd0122</code></a> Update dependabot.yml (<a href="https://github.com/aio-libs/aiohttp/issues/7888">#7888</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/41a9f1f5b9b2630b1f4b971811c7ef8f016262fb"><code>41a9f1f</code></a> Bump mypy from 1.7.0 to 1.7.1 (<a href="https://github.com/aio-libs/aiohttp/issues/7882">#7882</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/a04970150c6ce9fda22c9f63d947845f79148b4c"><code>a049701</code></a> Fix usage of proxy.py in test_proxy_functional (<a href="https://github.com/aio-libs/aiohttp/issues/7773">#7773</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7876">#7876</a>)</li> <li>Additional commits viewable in <a href="https://github.com/aio-libs/aiohttp/compare/v3.9.0...v3.9.1">compare view</a></li> </ul> </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aiohttp&package-manager=pip&previous-version=3.9.0&new-version=3.9.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.6 to 3.9.1. <details> <summary>Release notes</summary> Sourced from <a href="https://github.com/aio-libs/aiohttp/releases">aiohttp's releases</a>. <blockquote> <h2>3.9.1</h2> <h2>Bugfixes</h2> <ul> <li> Fixed importing aiohttp under PyPy on Windows. (<a href="https://github.com/aio-libs/aiohttp/issues/7848">#7848</a>) </li> <li> Fixed async concurrency safety in websocket compressor. (<a href="https://github.com/aio-libs/aiohttp/issues/7865">#7865</a>) </li> <li> Fixed <code>ClientResponse.close()</code> releasing the connection instead of closing. (<a href="https://github.com/aio-libs/aiohttp/issues/7869">#7869</a>) </li> <li> Fixed a regression where connection may get closed during upgrade. -- by :user:<code>Dreamsorcerer</code> (<a href="https://github.com/aio-libs/aiohttp/issues/7879">#7879</a>) </li> <li> Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:<code>Dreamsorcerer</code> (<a href="https://github.com/aio-libs/aiohttp/issues/7895">#7895</a>) </li> </ul> <hr /> <h2>3.9.0</h2> <h2>Features</h2> <ul> <li> Introduced <code>AppKey</code> for static typing support of <code>Application</code> storage. See <a href="https://docs.aiohttp.org/en/stable/web_advanced.html#application-s-config">https://docs.aiohttp.org/en/stable/web_advanced.html#application-s-config</a> (<a href="https://github.com/aio-libs/aiohttp/issues/5864">#5864</a>) </li> <li> Added a graceful shutdown period which allows pending tasks to complete before the application's cleanup is called. The period can be adjusted with the <code>shutdown_timeout</code> parameter. -- by :user:<code>Dreamsorcerer</code>. See <a href="https://docs.aiohttp.org/en/latest/web_advanced.html#graceful-shutdown">https://docs.aiohttp.org/en/latest/web_advanced.html#graceful-shutdown</a> (<a href="https://github.com/aio-libs/aiohttp/issues/7188">#7188</a>) </li> <li> Added <code>handler_cancellation <https://docs.aiohttp.org/en/stable/web_advanced.html#web-handler-cancellation></code>_ parameter to cancel web handler on client disconnection. -- by :user:<code>mosquito</code> This (optionally) reintroduces a feature removed in a previous release. Recommended for those looking for an extra level of protection against denial-of-service attacks. (<a href="https://github.com/aio-libs/aiohttp/issues/7056">#7056</a>) </li> <li> Added support for setting response header parameters <code>max_line_size</code> and <code>max_field_size</code>. </li> </ul>  </blockquote> ... (truncated) </details> <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst">aiohttp's changelog</a>. <blockquote> <h1>3.9.1 (2023-11-26)</h1> <h2>Bugfixes</h2> <ul> <li> Fixed importing aiohttp under PyPy on Windows. <code>[#7848](aio-libs/aiohttp#7848) <https://github.com/aio-libs/aiohttp/issues/7848></code>_ </li> <li> Fixed async concurrency safety in websocket compressor. <code>[#7865](aio-libs/aiohttp#7865) <https://github.com/aio-libs/aiohttp/issues/7865></code>_ </li> <li> Fixed <code>ClientResponse.close()</code> releasing the connection instead of closing. <code>[#7869](aio-libs/aiohttp#7869) <https://github.com/aio-libs/aiohttp/issues/7869></code>_ </li> <li> Fixed a regression where connection may get closed during upgrade. -- by :user:<code>Dreamsorcerer</code> <code>[#7879](aio-libs/aiohttp#7879) <https://github.com/aio-libs/aiohttp/issues/7879></code>_ </li> <li> Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:<code>Dreamsorcerer</code> <code>[#7895](aio-libs/aiohttp#7895) <https://github.com/aio-libs/aiohttp/issues/7895></code>_ </li> </ul> <hr /> <h1>3.9.0 (2023-11-18)</h1> <h2>Features</h2> <ul> <li> Introduced <code>AppKey</code> for static typing support of <code>Application</code> storage. See <a href="https://docs.aiohttp.org/en/stable/web_advanced.html#application-s-config">https://docs.aiohttp.org/en/stable/web_advanced.html#application-s-config</a> <code>[#5864](aio-libs/aiohttp#5864) <https://github.com/aio-libs/aiohttp/issues/5864></code>_ </li> <li> Added a graceful shutdown period which allows pending tasks to complete before the application's cleanup is called. The period can be adjusted with the <code>shutdown_timeout</code> parameter. -- by :user:<code>Dreamsorcerer</code>. See <a href="https://docs.aiohttp.org/en/latest/web_advanced.html#graceful-shutdown">https://docs.aiohttp.org/en/latest/web_advanced.html#graceful-shutdown</a> <code>[#7188](aio-libs/aiohttp#7188) <https://github.com/aio-libs/aiohttp/issues/7188></code>_ </li> <li> Added <code>handler_cancellation <https://docs.aiohttp.org/en/stable/web_advanced.html#web-handler-cancellation></code>_ parameter to cancel web handler on client disconnection. -- by :user:<code>mosquito</code> This (optionally) reintroduces a feature removed in a previous release. </li> </ul>  </blockquote> ... (truncated) </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aio-libs/aiohttp/commit/6333c026422c6b0fe57ff63cde4104e2d00f47f4"><code>6333c02</code></a> Release v3.9.1 (<a href="https://github.com/aio-libs/aiohttp/issues/7911">#7911</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/9dbd273093d6af6f5e1481816b05a7192860b440"><code>9dbd273</code></a> [PR <a href="https://github.com/aio-libs/aiohttp/issues/7673">#7673</a>/aa7d1a8f backport][3.9] Document release process (<a href="https://github.com/aio-libs/aiohttp/issues/7909">#7909</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/dd175b6b89564dc74fba0692a8a5f9a9b38e528a"><code>dd175b6</code></a> Fix regression with connection upgrade (<a href="https://github.com/aio-libs/aiohttp/issues/7879">#7879</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7908">#7908</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/946523d6380bd79e13146557432f46f6f9bbd53f"><code>946523d</code></a> Fix flaky websocket test (<a href="https://github.com/aio-libs/aiohttp/issues/7902">#7902</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7904">#7904</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/ddc2a26c9e0c43fd1229e4424f2a30d1b10ced13"><code>ddc2a26</code></a> [PR <a href="https://github.com/aio-libs/aiohttp/issues/7896">#7896</a>/9a7cfe77 backport][3.9] Fix some flaky tests (<a href="https://github.com/aio-libs/aiohttp/issues/7900">#7900</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/2ae4d6ffdd015f622bfb75dee98ad629240cccc4"><code>2ae4d6f</code></a> Message is not upgraded if Upgrade header is missing (<a href="https://github.com/aio-libs/aiohttp/issues/7895">#7895</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7898">#7898</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/bb111012706d3ef9edc525be3d8d4df410ad847f"><code>bb11101</code></a> Restore async concurrency safety to websocket compressor (<a href="https://github.com/aio-libs/aiohttp/issues/7865">#7865</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7889">#7889</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/6dd0122417f00ed4b2b353226a1b164b6463a245"><code>6dd0122</code></a> Update dependabot.yml (<a href="https://github.com/aio-libs/aiohttp/issues/7888">#7888</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/41a9f1f5b9b2630b1f4b971811c7ef8f016262fb"><code>41a9f1f</code></a> Bump mypy from 1.7.0 to 1.7.1 (<a href="https://github.com/aio-libs/aiohttp/issues/7882">#7882</a>)</li> <li><a href="https://github.com/aio-libs/aiohttp/commit/a04970150c6ce9fda22c9f63d947845f79148b4c"><code>a049701</code></a> Fix usage of proxy.py in test_proxy_functional (<a href="https://github.com/aio-libs/aiohttp/issues/7773">#7773</a>) (<a href="https://github.com/aio-libs/aiohttp/issues/7876">#7876</a>)</li> <li>Additional commits viewable in <a href="https://github.com/aio-libs/aiohttp/compare/v3.8.6...v3.9.1">compare view</a></li> </ul> </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aiohttp&package-manager=pip&previous-version=3.8.6&new-version=3.9.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

bdraco mentioned this pull request Nov 22, 2023

Ensure separate ZLibCompressor instances per coroutine when using max_sync_chunk_size #7862

Closed

5 tasks

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Nov 22, 2023

bdraco commented Nov 22, 2023

View reviewed changes

tests/test_websocket_writer.py Outdated Show resolved Hide resolved

bdraco force-pushed the safe_compress branch from 557812f to 19f2214 Compare November 22, 2023 11:42

bdraco changed the title ~~WIP: Restore async concurrency safety to websocket compressor~~ Restore async concurrency safety to websocket compressor Nov 22, 2023

bdraco force-pushed the safe_compress branch from 19f2214 to 6574aa6 Compare November 22, 2023 11:49

Restore async concurrency safety to websocket compressor

9a5274f

bdraco force-pushed the safe_compress branch from 6574aa6 to 9a5274f Compare November 22, 2023 11:50

bdraco marked this pull request as ready for review November 22, 2023 11:54

bdraco requested a review from asvetlov as a code owner November 22, 2023 11:54

bdraco added 3 commits November 22, 2023 17:12

mix messages

8e0bcbf

always lock

be83e37

always lock

f418d48

Dreamsorcerer reviewed Nov 22, 2023

View reviewed changes

tests/test_websocket_writer.py Outdated Show resolved Hide resolved

bdraco and others added 8 commits November 22, 2023 17:27

always lock, parametrize

fed5f4f

Update tests/test_websocket_writer.py

c70caf5

Co-authored-by: Sam Bull <git@sambull.org>

remove now that its parametrized

72ff923

make sure they are all still different so we know if we are mixing th…

cb122bd

…em up

make sure they are all still different so we know if we are mixing th…

06a7a0e

…em up

make sure they are all still different so we know if we are mixing th…

8aa6d01

…em up

parametrize 3rd test

b82d552

save some indent

f6160b6

Dreamsorcerer reviewed Nov 22, 2023

View reviewed changes

aiohttp/http_websocket.py Outdated Show resolved Hide resolved

create lock with compressor

f9a26b8

Dreamsorcerer mentioned this pull request Nov 23, 2023

🧪 Collect Cython coverage in CI aio-libs/yarl#961

Merged

Dreamsorcerer reviewed Nov 23, 2023

View reviewed changes

aiohttp/http_websocket.py Outdated Show resolved Hide resolved

bdraco and others added 6 commits November 24, 2023 06:46

revert much of the change and move the lock to the compressor

4025575

add comments to avoid breaking this in the future

cedc1ba

add comments to avoid breaking this in the future

20fc16d

add comments to avoid breaking this in the future

8ed0fb5

Merge branch 'master' into safe_compress

3c71810

Unneeded

bfc4405

Dreamsorcerer added backport-3.9 labels Nov 24, 2023

Dreamsorcerer enabled auto-merge (squash) November 24, 2023 13:36

empty commit to restart ci since pypy test is flakey

6d89a7f

Dreamsorcerer merged commit 86a2396 into aio-libs:master Nov 24, 2023
29 of 31 checks passed

bdraco deleted the safe_compress branch November 24, 2023 18:52

bdraco restored the safe_compress branch November 24, 2023 18:52

bdraco added a commit to bdraco/aiohttp that referenced this pull request Nov 24, 2023

Restore async concurrency safety to websocket compressor (aio-libs#7865)

b548b4e

Fixes aio-libs#7859 (cherry picked from commit 86a2396)

bdraco added a commit to bdraco/aiohttp that referenced this pull request Nov 24, 2023

Restore async concurrency safety to websocket compressor (aio-libs#7865)

dcfeba7

Fixes aio-libs#7859 (cherry picked from commit 86a2396)

Dreamsorcerer pushed a commit that referenced this pull request Nov 24, 2023

Restore async concurrency safety to websocket compressor (#7865) (#7890)

82c93ff

Fixes #7859 (cherry picked from commit 86a2396)

Dreamsorcerer pushed a commit that referenced this pull request Nov 24, 2023

Restore async concurrency safety to websocket compressor (#7865) (#7889)

bb11101

Fixes #7859 (cherry picked from commit 86a2396)

Dreamsorcerer mentioned this pull request Nov 26, 2023

Add a test case for #7859 #7866

Closed

5 tasks

bdraco deleted the safe_compress branch November 27, 2023 04:41

xiangxli pushed a commit to xiangxli/aiohttp that referenced this pull request Dec 4, 2023

Restore async concurrency safety to websocket compressor (aio-libs#7865)

a0af4a3

Fixes aio-libs#7859

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restore async concurrency safety to websocket compressor #7865

Restore async concurrency safety to websocket compressor #7865

bdraco commented Nov 22, 2023 •

edited

Loading

codecov bot commented Nov 22, 2023 •

edited

Loading

bdraco commented Nov 22, 2023

bdraco commented Nov 22, 2023

bdraco commented Nov 22, 2023

bdraco commented Nov 22, 2023

Dreamsorcerer commented Nov 22, 2023

Dreamsorcerer commented Nov 23, 2023

patchback bot commented Nov 24, 2023 •

edited

Loading

patchback bot commented Nov 24, 2023 •

edited

Loading

Restore async concurrency safety to websocket compressor #7865

Restore async concurrency safety to websocket compressor #7865

Conversation

bdraco commented Nov 22, 2023 • edited Loading

What do these changes do?

Are there changes in behavior for the user?

Related issue number

Checklist

codecov bot commented Nov 22, 2023 • edited Loading

Codecov Report

bdraco commented Nov 22, 2023

bdraco commented Nov 22, 2023

bdraco commented Nov 22, 2023

bdraco commented Nov 22, 2023

Dreamsorcerer commented Nov 22, 2023

Dreamsorcerer commented Nov 23, 2023

patchback bot commented Nov 24, 2023 • edited Loading

Backport to 3.9: 💔 cherry-picking failed — conflicts found

Backporting merged PR #7865 into master

patchback bot commented Nov 24, 2023 • edited Loading

Backport to 3.10: 💔 cherry-picking failed — conflicts found

Backporting merged PR #7865 into master

bdraco commented Nov 22, 2023 •

edited

Loading

codecov bot commented Nov 22, 2023 •

edited

Loading

patchback bot commented Nov 24, 2023 •

edited

Loading

patchback bot commented Nov 24, 2023 •

edited

Loading