Skip to content

Commit

Permalink
Remove unused kv_cache argument to public API
Browse files Browse the repository at this point in the history
  • Loading branch information
rlouf committed Feb 16, 2024
1 parent f1d2f78 commit bc71b23
Showing 1 changed file with 0 additions and 10 deletions.
10 changes: 0 additions & 10 deletions outlines/generate/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,6 @@ def __call__(
max_tokens: Optional[int] = None,
stop_at: Optional[Union[str, List[str]]] = None,
rng: Optional[torch.Generator] = None,
kv_cache: Optional[torch.tensor] = None,
) -> Union[str, List[str], List[List[str]]]:
"""Generate the full text sequence.
Expand All @@ -144,10 +143,6 @@ def __call__(
(per prompt)
stop_at
A string or list of strings at which the text generated will stop
kv_cache
A tensor containing the past key-value cache. It can be for instance
used when we are interleaving prompting and model calls. Defaults to
`None`.
rng
The random number generator. Defaults to a non-seeded `torch.Generator`
instance.
Expand Down Expand Up @@ -248,7 +243,6 @@ def stream(
max_tokens: Optional[int] = None,
stop_at: Optional[Union[str, List[str]]] = None,
rng: Optional[torch.Generator] = None,
kv_cache: Optional[torch.tensor] = None,
) -> Iterator[Union[List[str], List[List[str]], str]]:
"""Generate the text sequence one token at a time.
Expand All @@ -266,10 +260,6 @@ def stream(
(per prompt)
stop_at
A string or list of strings at which the text generated will stop
kv_cache
A tensor containing the past key-value cache. It can be for instance
used when we are interleaving prompting and model calls. Defaults to
`None`.
rng
The random number generator. Defaults to a non-seeded `torch.Generator`
instance.
Expand Down

0 comments on commit bc71b23

Please sign in to comment.