Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add C++ and Python API for Kokoro TTS models. #1715

Merged
merged 8 commits into from
Jan 16, 2025

Conversation

csukuangfj
Copy link
Collaborator

@csukuangfj csukuangfj commented Jan 16, 2025

Generated samples

0 af 1 af_bella 2 af_nicole
https://github.com/user-attachments/assets/6d0c0cb4-6b5b-48f4-a9e4-1eaaab86327a https://github.com/user-attachments/assets/70596402-a74f-4775-abcb-1f2b4fcaba60 https://github.com/user-attachments/assets/c76cf8fc-efa7-4f03-b356-7e1759eebea8
3 af_sarah 4 af_sky 5 am_adam
https://github.com/user-attachments/assets/3f160a11-05c4-47ed-b356-7a160e836ecc https://github.com/user-attachments/assets/1443e0a0-1c92-4ac2-9e5d-56c0a0dbf4af https://github.com/user-attachments/assets/153abc1a-421d-4e08-b62b-9fcb401fbfec
6 am_michael 7 bf_emma 8 bf_isabella
https://github.com/user-attachments/assets/6372c7d5-a2d5-4b8c-9e56-7f9957f592cc https://github.com/user-attachments/assets/6e9ccccd-de74-43e4-8594-f44121277d7a https://github.com/user-attachments/assets/5f56e847-6527-4eaf-be96-e2c9ad04366b
9 bm_george 10 bm_lewis
https://github.com/user-attachments/assets/54edffe5-9264-42ed-bc71-7de94f724706 https://github.com/user-attachments/assets/f4d33446-38c7-420d-bb1c-852203f18443
kokoro-0.mov
kokoro-1.mov
kokoro-10.mov
kokoro-9.mov
kokoro-8.mov
kokoro-7.mov
kokoro-6.mov
kokoro-5.mov
kokoro-4.mov
kokoro-3.mov
kokoro-2.mov

Usage with C++ binaries

curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kokoro-en-v0_19.tar.bz2
tar xf kokoro-en-v0_19.tar.bz2
rm kokoro-en-v0_19.tar.bz2

# mapping of sid to voice name
# 0->af, 1->af_bella, 2->af_nicole, 3->af_sarah, 4->af_sky, 5->am_adam
# 6->am_michael, 7->bf_emma, 8->bf_isabella, 9->bm_george, 10->bm_lewis

for sid in $(seq 0 10); do
  ./build/bin/sherpa-onnx-offline-tts \
    --debug=1 \
    --kokoro-model=./kokoro-en-v0_19/model.onnx \
    --kokoro-voices=./kokoro-en-v0_19/voices.bin \
    --kokoro-tokens=./kokoro-en-v0_19/tokens.txt \
    --kokoro-data-dir=./kokoro-en-v0_19/espeak-ng-data \
    --num-threads=2 \
    --sid=$sid \
    --output-filename="./tts/kokoro-$sid.wav" \
    "Today as always, men fall into two groups: slaves and free men. Whoever does not have two-thirds of his day for himself, is a slave, whatever he may be  a statesman, a businessman, an official, or a scholar."
done

Usage with Python APIs

curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kokoro-en-v0_19.tar.bz2
tar xf kokoro-en-v0_19.tar.bz2
rm kokoro-en-v0_19.tar.bz2

# mapping of sid to voice name
# 0->af, 1->af_bella, 2->af_nicole, 3->af_sarah, 4->af_sky, 5->am_adam
# 6->am_michael, 7->bf_emma, 8->bf_isabella, 9->bm_george, 10->bm_lewis

for sid in $(seq 0 10); do
  python3 ./python-api-examples/offline-tts.py \
    --debug=1 \
    --kokoro-model=./kokoro-en-v0_19/model.onnx \
    --kokoro-voices=./kokoro-en-v0_19/voices.bin \
    --kokoro-tokens=./kokoro-en-v0_19/tokens.txt \
    --kokoro-data-dir=./kokoro-en-v0_19/espeak-ng-data \
    --num-threads=2 \
    --sid=$sid \
    --output-filename="./tts/kokoro-$sid.wav" \
    "Today as always, men fall into two groups: slaves and free men. Whoever does not have two-thirds of his day for himself, is a slave, whatever he may be  a statesman, a businessman, an official, or a scholar."
done

@csukuangfj
Copy link
Collaborator Author

CC @thewh1teagle @giannik

Fixes #1679

@csukuangfj csukuangfj merged commit ffc6b48 into k2-fsa:master Jan 16, 2025
149 of 208 checks passed
@csukuangfj csukuangfj deleted the cpp-kokoro branch January 16, 2025 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant