Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enable more workers in api.py #621

Merged
merged 7 commits into from
Oct 18, 2024
Merged

Conversation

AnyaCoder
Copy link
Collaborator

@AnyaCoder AnyaCoder commented Oct 18, 2024

Is this PR adding new feature or fix a BUG?

Add feature / Fix BUG.

----

  1. Fix the problem that uvicorn can't start multiple workers, now support api to start multiple workers, increase throughput to achieve concurrent operation. Throughput is limited by GPU core count and utilisation. Sample command in the project root directory: python -m tools.api --compile --workers 4
  2. Support for caching the results of reference audio encoding in memory, for cases where a single speaker does not change it for a long time, reducing latency. This parameter adds a field for use_memory_cache to the request parameter of post_api. If use_memory_cache is set to on-demand, then the cache will always be used after the first encoding, regardless of any subsequent reference audio. If you want to change the reference audio, you need to set it to never and then reassign the reference audio. Default value never.
  3. Fix counter display error.

----

  1. 修复uvicorn多worker无法启动的问题,现在支持api启动多个workers,加大吞吐量实现并发操作。吞吐量受GPU核心数和利用率限制。在项目根目录下的示例命令:python -m tools.api --compile --workers 4
  2. 支持参考音频编码后的结果缓存在内存中,适用于单一说话人长时间不更改的情况,降低延迟。这个参数在post_api的请求参数基础上增加了一个use_memory_cache的字段。若use_memory_cache设置为on-demand, 则第一次编码后会一直使用该缓存,无论后来是否有其他参考音频。如果要修改参考音频,则需要将其设置为never后,重新指定参考音频。默认值never.
  3. 修复计数君显示错误。

@leng-yue leng-yue merged commit f15d9f2 into fishaudio:main Oct 18, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants