Fix: Speed up CSV generation by minimizing DB probes #6577
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bshankar
force-pushed
the
download-csv-optimization
branch
3 times, most recently
from
c293cb4
to
f43feab
Compare
bshankar
force-pushed
the
download-csv-optimization
branch
from
30c075d
to
e9b84d4
Compare
|
|
I just tested this on my local setup (2700ish projects), the download was success for all projects and just below timeout. For prod this should be even lower. |
|
@ramyaragupathy can be merged. |
prabinoid
approved these changes
Oct 1, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this? (check all applicable)
Describe this PR
Download CSV feature was unreasonably slow when the number of projects was high - 8 minutes for 2700 projects.
I moved the expensive DB calls outside the loop into just one query. It should be much faster now.
Review Guide
Use an DB with multiple users and lots of projects and task history. Preferably 2000+ projects in the search results. Press the download CSV button. Look for the right contributor count and the CSV file getting downloaded without a gateway timeout.