Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: reduce ttft and itl with stream_interval 1 #658

Merged
merged 1 commit into from
Jul 19, 2024
Merged

perf: reduce ttft and itl with stream_interval 1 #658

merged 1 commit into from
Jul 19, 2024

Conversation

zhyncs
Copy link
Member

@zhyncs zhyncs commented Jul 19, 2024

Motivation

as titled cc @merrymercy @Ying1123 @hnyls2002

Modification

as titled

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@Ying1123 Ying1123 merged commit ac971ff into sgl-project:main Jul 19, 2024
1 check passed
@zhyncs zhyncs deleted the interval branch July 19, 2024 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants