Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

return all tokens at once #189

Closed
lichangW opened this issue Aug 1, 2023 · 2 comments
Closed

return all tokens at once #189

lichangW opened this issue Aug 1, 2023 · 2 comments
Assignees
@AllentDan

Comments

@lichangW
Copy link

lichangW commented Aug 1, 2023

Motivation

Hi, we want an interface to return all generated tokens at once but not in stream style, is there an example or document? Actually, we need a restful api from which we can put setting and prompt in requests and get all output tokens in one response. thanks for help!

Related resources

No response

Additional context

No response
@lvhan028
Copy link
Collaborator

lvhan028 commented Aug 3, 2023

Working on it.

@AllentDan AllentDan mentioned this issue Aug 11, 2023
Merged
@AllentDan
Copy link
Collaborator

The restful APIs were added. For more details please refer to https://github.com/InternLM/lmdeploy/blob/main/docs/en/restful_api.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AllentDan AllentDan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lvhan028 @lichangW @AllentDan