Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Support streamWithThink for other models #6128

Open
RYQ-22 opened this issue Feb 2, 2025 · 10 comments
Open

[Feature Request] Support streamWithThink for other models #6128

RYQ-22 opened this issue Feb 2, 2025 · 10 comments
Labels
enhancement New feature or request planned planned feature, will support in the future

Comments

@RYQ-22
Copy link

RYQ-22 commented Feb 2, 2025

🥰 Feature Description

Currently, only the deepseek-reasoner model supports streamWithThink. However, when we use a third-party DeepSeek API through the OpenAI interface, streamWithThink is disabled.

Image

Image

🧐 Proposed Solution

I have a naive idea but I'm not sure if it‘ll work: detect <think> in stream for all models and turn on "think" mode automatically.

📝 Additional Information

No response

@RYQ-22 RYQ-22 added the enhancement New feature or request label Feb 2, 2025
@ninirobot
Copy link

Image
能否也支持一下谷歌模型的思考过程?谢谢!

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


! [Image] (https://github.com/user-attachments/aSSETS/E037AB30-5F53-410d-b74b-a537ca019612))
Can you also support the thinking process of Google model? Thanks!

@Leizhenpeng Leizhenpeng added the planned planned feature, will support in the future label Feb 4, 2025
@jackalcooper
Copy link
Contributor

related: #6141

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


related: #6141

@jackalcooper
Copy link
Contributor

#6186 修复了格式

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Fixed format at #6186

@xzcxzcyy
Copy link

xzcxzcyy commented Feb 8, 2025

#6186 修复了格式

可以做成一个通用的方案吗?现在看起来只对SiliconFlow生效。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Fixed format in #6186

Can it be made into a general solution? Now it looks like it only works for SiliconFlow.

@jackalcooper
Copy link
Contributor

jackalcooper commented Feb 8, 2025

#6186 修复了格式

可以做成一个通用的方案吗?现在看起来只对SiliconFlow生效。

因为只有openai原版api带有的的规范变成通用的自定义endpoint才比较合适,现在这个field是deepseek 特有的,所以不太好一口气全改(比如有些模型输出有可能是<think>)。现在的代码本身是通用的,每个模型实现自己的parseSSE函数。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Fixed format in #6186

Can it be made into a general solution? Now it looks like it only works for SiliconFlow.

Because it is more appropriate to only the specifications included in the original openai API become general custom endpoints, this field is now unique to deepseek, so it is not easy to modify them all in one go (for example, some model output may be ). The code now is general in itself, and each model implements its own parseSSE function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request planned planned feature, will support in the future
Projects
None yet
Development

No branches or pull requests

6 participants