Melting Away

[pzelasko]

New corpora

• GigaSpeech (The GigaSpeech recipe #283, thanks @jimbozhang)
• Dihard 3 (Dihard3 recipe (and other minor changes) #287, thanks @desh2608)
• GALE Arabic and Mandarin (GALE Arabic and Mandarin recipes #296, thanks @desh2608)
• CMU and CSLU Kids (CMU and CSLU Kids speech recipes #297, thanks @desh2608)
• MTedX (Mtedx #301, thanks @m-wiesner)
• LibriTTS (LibriTTS recipe #306)

New features

• Reading huge manifests lazily with Apache Arrow (documentation and examples are coming) ([experimental] Lazy manifest loading for huge datasets #286, Lazy manifests usable with a non-shuffled sampler + dataset #288, Automatically inferring the schema for direct Arrow serialization #289, Handle empty supervisions serialization with pyarrow #290, Pickling support for LazyDict + minor fixes #292, On-the-fly cut filtering + improvements in arrow/lazy-dict  #294)
• Sequential JSONL writer storing manifests on disk as they are created (Sequential JSONL writer #302)
• Support for alignments in SupervisionSegment (Adding alignments to supervision #304, Generating masks using alignments #310, Remove recursive serialization for alignment #313, thanks @desh2608)
• PyTorch Kaldi-compatible feature extractors that support GPU, batching and autograd (Kaldi feature extractor layers (kudos to Jesus Villalba) #307, thanks @jesus-villalba)
• Reading, writing, and uploading features to URLs (e.g. S3 or GCP) (Support for reading, writing, and uploading features to URLs (HTTP, S3, GCP, Azure, etc.) #312)
• Store waveforms of cuts as audio recordings to disk (Store waveforms of cuts as audio recordings to disk #316, thanks @entn-at)
• Support for importing Kaldi's feats.scp and reading features directly from scp/ark (Support for importing Kaldi's feats.scp and reading features directly from scp/ark #318)

General improvements

• add multi thread to process AIShell data (add multi thread to process AIShell data #259, thanks @pingfengluo)
• tracking dev versions (add git commit hash to version in form X.Y.Z.dev-gitgitsha-indexclean #291, thanks @oplatek)
• Explicitly set UTF-8 encoding when reading README.md in setup.py (Explicitly set UTF-8 encoding when reading README.md in setup.py #293, thanks @entn-at)
• Auto-add link to source code in docs (Auto-add link to source code in docs #295)
• cut.resample() (Add option to resample cuts #299)
• fixing flaky tests (Fix flaky smoothing tests #300)
• fix AMI CLI mode (Fixed AMI CLI mode #303, thanks @desh2608)
• handle zero energy error in audio mixing (Handle zero energy error in audio mixing #305)
• update Kaldi related docs (Update feature/Kaldi related docs a bit #308)
• add a missing SpecAugment parameter (Add a missing SpecAugment parameter #309)
• fixing edge cases for audio transforms (Fix edge cases when combining speed perturb, resampling, and cutting … #311)
• Add drop_last option in *Set.split() (Add drop_last option in *Set.split() #315)
• Support h5py file modes in feature writers (Support h5py file modes in feature writers #317)
• don't using kaldi reco2dur and fix some error in bin/lhotse (Support for importing Kaldi's feats.scp and reading features directly from scp/ark #318, thanks @shanguanma)
• Fix cut num of samples bug (Fix cut num of samples bug #322, thanks @dophist)
• use whitespace in kaldi field-splitting (use whitespace in kaldi field-splitting #323, thanks @dophist)

---

This discussion was created from the release Melting Away.