Split out model vs microbatch execution #10737
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…l execution In commit 8fe5ea1 the adding of the relationship to the adapter cache got moved to _before_ the post hook during model execution. This was done because of some wonkiness of having the microbatch model and model execution logic in the same place. Now that we've split the relevant logic, we can move the caching back to it's original spot in the order of operations. We don't think this actually changes anything, but we're moving it back juuuust in case because this is a very hot code path.
QMalcolm
force-pushed
the
qmalcolm--split-out-model-vs-microbatch-execution
branch
from
dfb5076
to
89f1c67
Compare
MichelleArk
approved these changes
Sep 18, 2024
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #10737 +/- ##
==========================================
- Coverage 89.00% 88.95% -0.05%
==========================================
Files 181 181
Lines 22964 22971 +7
==========================================
- Hits 20440 20435 -5
- Misses 2524 2536 +12
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
The failing unit test, |
QMalcolm
deleted the
qmalcolm--split-out-model-vs-microbatch-execution
branch
peterallenwebb
pushed a commit
that referenced
this pull request
Sep 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #N/A
Description
In #10677 we added microbatch model execution logic to the
ModelRunner.execute. In doing so we changed the order of operations of some normal model execution steps. This PR splits the logic so that we can return the order of operations for normal model execution back to what it was previously while still supporting microbatch model execution. We don't know if the change of order operations that happened in #10677 was problematic, but moving it back is being done out caution.Checklist