Stream functions and their properties. #337
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces support for
promptStreaming
.When you use
promptStreaming
, you get back a stream where you can collect each of the individual properties of a type as they are streamed from the LLM. Finally, you get the final result serialized.This PR also includes some fixes to the
@Description
annotations so they work properly in Kotlin, which broke after we fixed the Scala and Java ones.The implementation for streaming functions looks like this:
CFunction
we ask the LLM to reply with.Example:
When we run this program each line below corresponds to an individual property being streamed then the final object is returned.