Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LibriCSS recipe #4321

Merged
merged 62 commits into from
Nov 13, 2020
Merged

LibriCSS recipe #4321

merged 62 commits into from
Nov 13, 2020

Conversation

desh2608
Copy link
Contributor

@desh2608 desh2608 commented Nov 4, 2020

This recipe contains multi-speaker diarization + ASR for long recording. There are 2 scenarios:

  1. s5_mono: The input is single-channel mixed recording (possibly containing overlapping speech).
  2. s5_css: The input is continuously separated audio streams. By "continuous" separation, we mean a method that performs separation on windowed segments, such that the same speaker may be split across different audio streams in different windows.

For diarization, we use x-vectors and spectral clustering. For ASR, we use a TDNNF-based model (pretrained model from Librispeech) fine-tuned on Librispeech + simulated RIRs. Evaluation is in terms of cpWER (similar to CHiME-6). More details are in the README.md file.

@kkm000 kkm000 self-requested a review November 10, 2020 14:46
Copy link
Contributor

@kkm000 kkm000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whew! That's quite a changeset!

I did not mark nits in files that otherwise needed no attention, not to drawn you in the changes. I'm sure I missed more than a couple things, but hope this would be already helpful.

egs/callhome_diarization/v1/diarization/vb_hmm_xvector.py Outdated Show resolved Hide resolved
egs/callhome_diarization/v1/diarization/vb_hmm_xvector.py Outdated Show resolved Hide resolved
egs/callhome_diarization/v1/diarization/vb_hmm_xvector.sh Outdated Show resolved Hide resolved
egs/libri_css/README.md Outdated Show resolved Hide resolved
egs/libri_css/s5_css/run.sh Outdated Show resolved Hide resolved
egs/libri_css/s5_css/run.sh Outdated Show resolved Hide resolved
egs/libri_css/s5_mono/local/decode.sh Outdated Show resolved Hide resolved
egs/libri_css/s5_mono/local/decode.sh Outdated Show resolved Hide resolved
@desh2608
Copy link
Contributor Author

@kkm000 Thanks this was very helpful! I have made (almost) all the changes you suggested.

@danpovey
Copy link
Contributor

danpovey commented Nov 12, 2020 via email

Copy link
Contributor

@kkm000 kkm000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few glitches introduced in the change.

egs/libri_css/s5_mono/local/best_wer_matching.py Outdated Show resolved Hide resolved
egs/libri_css/s5_mono/local/get_perspeaker_output.py Outdated Show resolved Hide resolved
egs/libri_css/s5_mono/local/download_diarizer.sh Outdated Show resolved Hide resolved
egs/libri_css/s5_mono/local/dscore.sh Outdated Show resolved Hide resolved
egs/libri_css/s5_mono/local/dscore.sh Outdated Show resolved Hide resolved
@kkm000
Copy link
Contributor

kkm000 commented Nov 12, 2020

@desh2608,

It's really incredible how much new bash tricks I learn from your reviews, thanks!

You make me 😳 :) By the way, there is a nice book called Pure Bash Bible. A lot of good tricks!

Copy link
Contributor

@kkm000 kkm000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping this note as a reminder: change the call to grep back to calling find(1) as external process, and we're done!

@jtrmal
Copy link
Contributor

jtrmal commented Nov 13, 2020 via email

@kkm000
Copy link
Contributor

kkm000 commented Nov 13, 2020

Thank you @desh2608, that was a helluva work! LGTM, merging.

@kkm000 kkm000 merged commit 1670662 into kaldi-asr:master Nov 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants