refactor: abusing goroutines and channel #1561
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
zijiren233
commented
Jun 26, 2024
zijiren233
commented
zijiren233
changed the title
|
可以写明一下具体更改了什么以及解决了什么问题吗。 |
|
重写没必要使用goroutine和channel的地方，从而提高性能 |
zijiren233
changed the title
|
之前的写法确实问题很大，但是这次改动这么大，最好能提供更多模型的测试案例，而且响应最好能稍微长一点，至少要超过一次 stream event 的长度。 |
|
这是SSE请求界面，一个接收消息就是一个event |
|
@zijiren233 参照 CI 改一下 commit 的格式吧。 |
|
已修复commit message |
songquanpeng
changed the title
common/utils.go
Outdated
| @@ -18,6 +18,7 @@ func LogQuota(quota int64) string { | |||
| } | |||
|
|
|||
| func RenderStringData(c *gin.Context, data string) { | |||
| data = strings.TrimPrefix(data, "data: ") | |||
There was a problem hiding this comment.
openai里面，是直接把获取到的string发送过去，这时候开头会有两个data:
| str = strings.TrimPrefix(str, "data: ") | ||
| str = strings.TrimSuffix(str, "\r") | ||
| c.Render(-1, common.CustomEvent{Data: "data: " + str}) | ||
| c.Writer.Flush() |
There was a problem hiding this comment.
SSE必须加flush，否则即使往conn里面写数据，客户端也不会立即获取到数据
可以看看c.Stream实现，其实就是简单的包装了下
|
我把 render 相关的抽到 render 包下了，除此之外没有其他改动 |
jinjianming
pushed a commit
to jinjianming/one-api
that referenced
this pull request
Jul 17, 2024
* refactor: abusing goroutines * fix: trim data prefix * refactor: move functions to render package * refactor: add back trim & flush --------- Co-authored-by: JustSong <quanpengsong@gmail.com>
mxdlzg
pushed a commit
to mxdlzg/one-api
that referenced
this pull request
Oct 15, 2024
* refactor: abusing goroutines * fix: trim data prefix * refactor: move functions to render package * refactor: add back trim & flush --------- Co-authored-by: JustSong <quanpengsong@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.