Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix double compress when compression enabled and compressed file exists #8014

Merged
merged 3 commits into from
Jan 11, 2024

Conversation

bdraco
Copy link
Member

@bdraco bdraco commented Jan 9, 2024

What do these changes do?

If the handler has .enable_compression set and we use the pre-compressed .gz we would compress twice. The caller may not know if the .gz file exists or not or have a mixed use case where some files are pre-compressed and some are compressed on the fly based if they have limited storage

fixes #8011

Are there changes in behavior for the user?

Files are no longer double compressed which is likely never the desired outcome

Related issue number

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
    • The format is <Name> <Surname>.
    • Please keep alphabetical order, the file is sorted by names.
  • Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<type> for example (588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the pr
    • ensure type is one of the following:
      • .feature: Signifying a new feature.
      • .bugfix: Signifying a bug fix.
      • .doc: Signifying a documentation improvement.
      • .removal: Signifying a deprecation or removal of public API.
      • .misc: A ticket has been closed, but it is not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

If the handler has `.enable_compression` set and we use the pre-compressed
.gz we would compress twice. The caller may not know if the `.gz` file
exists or not or have a mixed use case where some files are
pre-compressed and some are compressed on the fly based if they have
limited storage

fixes #8011
@bdraco bdraco force-pushed the fix_double_compression branch from 57a4c94 to 6c11675 Compare January 9, 2024 22:42
@psf-chronographer psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Jan 9, 2024
Copy link

codecov bot commented Jan 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5f699bb) 97.46% compared to head (0965604) 97.46%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #8014   +/-   ##
=======================================
  Coverage   97.46%   97.46%           
=======================================
  Files         107      107           
  Lines       32529    32548   +19     
  Branches     3790     3790           
=======================================
+ Hits        31703    31722   +19     
  Misses        624      624           
  Partials      202      202           
Flag Coverage Δ
CI-GHA 97.37% <100.00%> (+<0.01%) ⬆️
OS-Linux 97.05% <100.00%> (+<0.01%) ⬆️
OS-Windows 95.55% <100.00%> (+<0.01%) ⬆️
OS-macOS 96.86% <100.00%> (+0.18%) ⬆️
Py-3.10.11 95.47% <100.00%> (+<0.01%) ⬆️
Py-3.10.13 96.85% <100.00%> (+<0.01%) ⬆️
Py-3.11.7 96.60% <100.00%> (+0.02%) ⬆️
Py-3.12.1 96.67% <100.00%> (+0.02%) ⬆️
Py-3.8.10 95.44% <100.00%> (+<0.01%) ⬆️
Py-3.8.18 96.79% <100.00%> (+<0.01%) ⬆️
Py-3.9.13 95.44% <100.00%> (+<0.01%) ⬆️
Py-3.9.18 96.82% <100.00%> (-0.01%) ⬇️
Py-pypy7.3.14 96.38% <100.00%> (+<0.01%) ⬆️
VM-macos 96.86% <100.00%> (+0.18%) ⬆️
VM-ubuntu 97.05% <100.00%> (+<0.01%) ⬆️
VM-windows 95.55% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bdraco bdraco closed this Jan 9, 2024
@bdraco bdraco reopened this Jan 9, 2024
@bdraco bdraco marked this pull request as ready for review January 9, 2024 22:55
@bdraco bdraco requested a review from asvetlov as a code owner January 9, 2024 22:55
@bdraco bdraco merged commit 92655a5 into master Jan 11, 2024
32 of 34 checks passed
@bdraco bdraco deleted the fix_double_compression branch January 11, 2024 00:56
Copy link
Contributor

patchback bot commented Jan 11, 2024

Backport to 3.9: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 92655a5 on top of patchback/backports/3.9/92655a50987e8bdfb6241dabbedc4c3cc8f55272/pr-8014

Backporting merged PR #8014 into master

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/aio-libs/aiohttp.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/3.9/92655a50987e8bdfb6241dabbedc4c3cc8f55272/pr-8014 upstream/3.9
  4. Now, cherry-pick PR Fix double compress when compression enabled and compressed file exists #8014 contents into that branch:
    $ git cherry-pick -x 92655a50987e8bdfb6241dabbedc4c3cc8f55272
    If it'll yell at you with something like fatal: Commit 92655a50987e8bdfb6241dabbedc4c3cc8f55272 is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x 92655a50987e8bdfb6241dabbedc4c3cc8f55272
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR Fix double compress when compression enabled and compressed file exists #8014 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/3.9/92655a50987e8bdfb6241dabbedc4c3cc8f55272/pr-8014
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Copy link
Contributor

patchback bot commented Jan 11, 2024

Backport to 3.10: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 92655a5 on top of patchback/backports/3.10/92655a50987e8bdfb6241dabbedc4c3cc8f55272/pr-8014

Backporting merged PR #8014 into master

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/aio-libs/aiohttp.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/3.10/92655a50987e8bdfb6241dabbedc4c3cc8f55272/pr-8014 upstream/3.10
  4. Now, cherry-pick PR Fix double compress when compression enabled and compressed file exists #8014 contents into that branch:
    $ git cherry-pick -x 92655a50987e8bdfb6241dabbedc4c3cc8f55272
    If it'll yell at you with something like fatal: Commit 92655a50987e8bdfb6241dabbedc4c3cc8f55272 is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x 92655a50987e8bdfb6241dabbedc4c3cc8f55272
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR Fix double compress when compression enabled and compressed file exists #8014 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/3.10/92655a50987e8bdfb6241dabbedc4c3cc8f55272/pr-8014
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

bdraco added a commit that referenced this pull request Jan 11, 2024
bdraco added a commit that referenced this pull request Jan 11, 2024
bdraco added a commit that referenced this pull request Jan 11, 2024
bdraco added a commit that referenced this pull request Jan 11, 2024
@steverep
Copy link
Member

steverep commented Jul 8, 2024

If the client makes a request for "/hello.txt.gz" and the handler enables compression, what should the expected response be? Is it intentional that that would still result in double compression?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bot:chronographer:provided There is a change note present in this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FileResponse + enable_compression + .gz will compress twice
3 participants