Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
The
latest_partition
function results shouldn't change from second to second, yet it could be called many times simultaneously if a dashboard has many charts dependent on the same table. Conservatively memoize the result of this function for 60 seconds to prevent an extra roundtrip to the Presto db. Perhaps this could be extended to 5 minutes in the future, but wanted to start out conservativelyTEST PLAN
CI, add print statements to the function and see that the function isn't called when run for the 2nd or 3rd time in a minute
ADDITIONAL INFORMATION
to: @villebro @john-bodley @betodealmeida