feat: Use X-Accel-Redirect to improve file download speed #1084
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of making FastAPI handle file download, which has serious performance issues on big files [1], this change uses nginx's `X-Accel` feature to delegate single-file downloads to nginx. Partial fix for #1079, as it solves the CPU usage issue for single-file downloads. [1] fastapi/fastapi#6050
gantoine
approved these changes
Aug 12, 2024
gantoine
force-pushed
the
feat/implement-x-accel-redirect
branch
3 times, most recently
from
ead6157
to
d20f4ad
Compare
gantoine
force-pushed
the
feat/implement-x-accel-redirect
branch
2 times, most recently
from
0038c05
to
9ece3eb
Compare
gantoine
force-pushed
the
feat/implement-x-accel-redirect
branch
from
9ece3eb
to
0eb5bba
Compare
adamantike
commented
Aug 16, 2024
gantoine
force-pushed
the
feat/implement-x-accel-redirect
branch
from
72872c7
to
dfe1483
Compare
gantoine
force-pushed
the
feat/implement-x-accel-redirect
branch
from
dfe1483
to
fed465a
Compare
adamantike
force-pushed
the
feat/implement-x-accel-redirect
branch
from
26b0dcd
to
4f5029f
Compare
gantoine
approved these changes
Aug 16, 2024
spiceratops
added a commit
to spiceratops/k8s-gitops
that referenced
this pull request
Sep 1, 2024
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [rommapp/romm](https://github.com/rommapp/romm) | minor | `3.4.0` -> `3.5.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>rommapp/romm (rommapp/romm)</summary> ### [`v3.5.0`](https://github.com/rommapp/romm/releases/tag/3.5.0) [Compare Source](https://github.com/rommapp/romm/compare/3.4.0...3.5.0) #### What's Changed - misc: Improve typing for feed schemas by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1064 - \[ROMM-1063] Add Amiga CD32 to emulatorjs game list by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1065 - fix: fab overlay fixed by [@​zurdi15](https://github.com/zurdi15) in [rommapp/romm#1073 - fix: Readable setup text on white theme by [@​zurdi15](https://github.com/zurdi15) in [rommapp/romm#1072 - fix: Allow access to Tinfoil feed when download auth is disabled by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1078 - fix: Set Gunicorn option forwarded-allow-ips by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1077 - Improve speed of fetching siblings for roms by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1076 - Ruffle flash emulator by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1049 - Calculate and store hashes for rom files by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1005 - Hotfix scans when running HASH_SCAN by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1081 - Fix playing emulatorjs + better platform icon loading by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1086 - Fix home icon size on safari/ios by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1088 - misc: Upgrade Python to v3.12 and Alpine to v3.20 by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1092 - feat: Use X-Accel-Redirect to improve file download speed by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1084 - misc: Pin Node version to v20 by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1095 - Add icon to gallery header for current platform by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1093 - Upload progress bars by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1096 - Skip compressed files if theyre invalid by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1097 - Bump emulatorjs to 4.1.1 by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1100 - feat: Use nginx mod_zip to generate multi-file zip downloads by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1102 - misc: Use single SQLAlchemy engine and session maker by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1105 - Switch funding to open collective by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1108 - Replace illegal fs chars in filenames by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1109 - Upload dialog hotfixed + clear button by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1118 - Fix detecting if platform is flash games by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1120 - feat(UX) - Allow links to open in a new tab by [@​SaraVieira](https://github.com/SaraVieira) in [rommapp/romm#1116 - fix(icons) - match icon names to platform names by [@​SaraVieira](https://github.com/SaraVieira) in [rommapp/romm#1122 - Disable auth on rom content get endpoint when env variable is present by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1125 - Add support for formatting vuejs with trunk by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1124 - Add known bios files from retropie project by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1126 - Add titledb field to tinfoil response by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1127 - Better performance for large collections by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1128 - \[ROMM-1113] Add file path in rom edit window by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1129 - misc: Tag Docker image with major version only by [@​adamantike](https://github.com/adamantike) in [rommapp/romm#1131 - build(deps): bump axios from 1.6.8 to 1.7.4 in /frontend by [@​dependabot](https://github.com/dependabot) in [rommapp/romm#1132 - Bump axios to 1.7.4 by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1134 - \[ROMM-1107] Add env variable to disable in-browser emulation by [@​gantoine](https://github.com/gantoine) in [rommapp/romm#1133 - Default both fast forward and rewind to enabled by [@​SaraVieira](https://github.com/SaraVieira) in [rommapp/romm#1136 - Allow user to unmatch rom by [@​SaraVieira](https://github.com/SaraVieira) in [rommapp/romm#1138 #### New Contributors - [@​SaraVieira](https://github.com/SaraVieira) made their first contribution in [rommapp/romm#1116 **Full Changelog**: rommapp/romm@3.4.0...3.5.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC41OS4yIiwidXBkYXRlZEluVmVyIjoiMzguNTkuMiIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsicmVub3ZhdGUvY29udGFpbmVyIiwidHlwZS9taW5vciJdfQ==-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of making FastAPI handle file download, which has serious performance issues on big files [1], this change uses nginx's
X-Accelfeature to delegate single-file downloads to nginx.Partial fix for #1079, as it solves the CPU usage issue for single-file downloads.
[1] fastapi/fastapi#6050