Fix iterator support for replicate.run() #383
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aron
force-pushed
the
fix-run-iterable
branch
2 times, most recently
from
69e3773 to
eb4c0df
Compare
zeke
reviewed
Oct 24, 2024
| } | ||
|
|
||
|
|
||
| def _version_with_schema(id: str = "v1", output_schema: Optional[object] = None): |
zeke
approved these changes
Oct 24, 2024
Prior to 1.0.0 `replicate.run()` would return an iterator for cog models that output a type of `Iterator[Any]`. This would poll the `predictions.get` endpoint for the in progress prediction and yield any new output. When implementing the new file interface we introduced two bugs: 1. The iterator didn't convert URLs returned by the model into `FileOutput` types making it inconsistent with the non-iterator interface. This is controlled by the `use_file_outputs` argument. 2. The iterator was returned without checking if we are using the new blocking API introduced by default and controlled by the `wait` argument. This commit fixes these two issues, consistently applying the `transform_output` function to the output of the iterator as well as returning the polling iterator (`prediciton.output_iterator`) if the blocking API has not successfully returned a completed prediction. The tests have been updated to exercise both of these code paths.
aron
force-pushed
the
fix-run-iterable
branch
from
eb4c0df to
6c427f7
Compare
nickstenning
approved these changes
Oct 28, 2024
There was a problem hiding this comment.
I've tested this locally and it fixes #381. Let's get it out and released.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prior to 1.0.0
replicate.run()would return an iterator for cog models that output a type ofIterator[Any]. This would poll thepredictions.getendpoint for the in progress prediction and yield any new output.When implementing the new file interface we introduced two bugs:
FileOutputtypes making it inconsistent with the non-iterator interface. This is controlled by theuse_file_outputsargument.waitargument.This commit fixes these two issues, consistently applying the
transform_outputfunction to the output of the iterator as well as returning the polling iterator (prediciton.output_iterator) if the blocking API has not successfully returned a completed prediction.The tests have been updated to exercise both of these code paths.