Code

WIP Sequence training of nnet3 models#3

Code

Closed

vimalmanohar wants to merge 159 commits intodanpovey:chaindanpovey/kaldi:chainfrom vimalmanohar:sequencevimalmanohar/kaldi:sequenceCopy head branch name to clipboard

+31,513-1,808

Commits

Commits on Dec 19, 2015

• sequence: Basic outline of header files for sequence training of nnet3

  vimalmanohar

  committed
  ac873d0

Commits on Dec 21, 2015

• sequencue: Further work on sequence training code

  vimalmanohar

  committed
  a146504

Commits on Dec 22, 2015

• sequencue: Added functions to split and merge supervisions

  vimalmanohar

  committed
  774d934

Commits on Dec 24, 2015

• sequence: Further additions to sequence training

  vimalmanohar

  committed
  43d1b22

Commits on Dec 26, 2015

• sequence: Added discriminative training functions

  vimalmanohar

  committed
  1fc4aa2

Commits on Dec 27, 2015

• sequence: Added binary files and modified lattice functions

  vimalmanohar

  committed
  a68f426

• sequence: Minor fix

  vimalmanohar

  committed
  d4fd87a

Commits on Dec 28, 2015

• sequence: nnet3-discriminative-train program added

  vimalmanohar

  committed
  332dbd2

Commits on Dec 30, 2015

• sequence: Modified some scripts to support new additions related to sequence training

  vimalmanohar

  committed
  81229bb

• sequence: Added scripts for discriminative training in nnet3

  vimalmanohar

  committed
  79f3e8e

• align_fix: Added --use-gpu option to nnet3-align-compiled

  vimalmanohar

  committed
  3c9680e

Commits on Jan 7, 2016

• changes regarding determinization of chain egs, merging from another branch

  danpovey

  committed
  7ebeacb

• Merged ApplySignum from snr_clean

  vimalmanohar

  committed
  6848f31

• jesus branch: adding debugged version of determinization changes, to fix memory-exhaustion issue found by Xiang Li

  danpovey

  committed
  fed777f

• improvement to steps/nnet3/chain/get_egs.sh script for speed and memory consumption

  danpovey

  committed
  e210ccb

• nnet3_to_dot.py : Added support for round descriptors and removed bug in sum descriptor processing

  vijayaditya

  committed
  8d342eb

• Merge pull request kaldi-asr#439 from vijayaditya/new_feature

  Show description for b5fb3f9
  danpovey

  committed
  b5fb3f9

• sequence: Fixed bug in Mpe variants

  vimalmanohar

  committed
  63ce5f3

• sequence: Fixed bugs in discriminative egs creation and added acoustic scale option

  vimalmanohar

  committed
  116a77d

• sequence: Added OtherStlVectorComparator in stl-utils.h

  vimalmanohar

  committed
  081d2fe

• sequence: steps/nnet3: Modifications to discriminative scripts

  vimalmanohar

  committed
  1989a70

• sequence: swbd/s5c/local: run_discriminative script minor modification

  vimalmanohar

  committed
  e97c558

• sequence: Adding extract-column, vector-to-feat from snr

  vimalmanohar

  committed
  2f31251

Commits on Jan 8, 2016

• jesus branch: merging changes from chain branch (mostly merged in turn from master)

  danpovey

  committed
  7e5b1aa

• Optimizations to cudamatrix library

  danpovey

  committed
  29258f2

• some fixes to previous commit regarding cudamatrix optimizations

  danpovey

  committed
  75b88b6

• master: merging most code changes from chain branch (but not the actual chain code)-- want to keep shared parts of the code the same.

  danpovey

  committed
  8df46b8

• minor documentation clarification

  danpovey

  committed
  460da6f

• cudamatrix: fixes to kernels that were using __syncthreads incorrectly (not sure if will fix a problem remi.fran6 was experiencing).

  danpovey

  committed
  bc13196

• minor documentation change: change recommended directory name.

  danpovey

  committed
  343a712

Commits on Jan 9, 2016

• documentation change: add entry to glossary

  danpovey

  committed
  8d1dcbd

• Add speed test for cudamatrix ApplyHeaviside that was missing

  danpovey

  committed
  1cc28da

• sequence: Major bug fix

  vimalmanohar

  committed
  93edd3c

• sequence: Support averaging in vector-sum because sometimes it is painful to do

  vimalmanohar

  committed
  6a04924

• Fixed typo in nnet3/decode.sh

  vimalmanohar

  committed
  3033536

• sequence: Minor changes to discriminative scripts

  vimalmanohar

  committed
  c225be1

• sequence: Added preliminary version of script to adjust priors

  vimalmanohar

  committed
  8ae4275

• Modification to configure script to touch cudamatrix/cu-common.h when it is called- will tend to reduce compilation errors due to changing from not using to using CUDA, or vice versa.

  danpovey

  committed
  d3c7778

Commits on Jan 10, 2016

• jesus branch: various new example scripts, and a couple new script options.

  danpovey

  committed
  ab26eca

• jesus branch: --extra-left-context-initial and --extra-right-context-final, making it possible to separately specify the extra context at the (start,end) of the file in nnet3 decoding.

  danpovey

  committed
  f9653dc

Commits on Jan 11, 2016

• Tabs to spaces

  Show description for c32b235
  

  Tim Hutt

  committed
  c32b235

• Add note about patch failing due to line endings

  Show description for cc50db7
  

  Tim Hutt

  committed
  cc50db7

• Don't convert patch file line endings to LF

  Show description for 1aac4b2
  

  Tim Hutt

  committed
  1aac4b2

• Removed note about git archive mangling patch line endings

  Show description for 40136da
  

  Tim Hutt

  committed
  40136da

• Don't mangle patch file line endings in all directories

  Show description for 50d8431
  

  Tim Hutt

  committed
  50d8431

• Change line endings in Windows patch file to CRLF

  Show description for 57e7f78
  

  Tim Hutt

  committed
  57e7f78

• bug-fix to previous commit about --extra-right-context-final, error in registering.

  danpovey

  committed
  7f00d46

• jesus branch: some example-script changes

  danpovey

  committed
  bf1cdfb

• fixes to chain derivative computation to prevent case when alphas become close to infinity, generating NaNs in derivatives

  danpovey

  committed
  623e05f

• fixes to chain derivative computation to prevent case when alphas become close to infinity, generating NaNs in derivatives

  danpovey

  committed
  6d3e499

Commits on Jan 12, 2016

• modify extract_ivectors_online.sh to properly respect --online-ivector-period option (fix to issue found by remi francis)

  danpovey

  committed
  b478057

• chain branch: merging changes from master

  danpovey

  committed
  ebb98fe

• fix to how CUDA block and grid sizes are computed for common operations, to work for larger matrices.

  danpovey

  committed
  155dbc0

• chain branch: merging upstream/master

  danpovey

  committed
  723275c

Commits on Jan 13, 2016

• Fix an overflow-related failure of chain training, identified by Xiang Li.

  danpovey

  committed
  22207d4

• script change [jesus branch: tuning-related]

  danpovey

  committed
  ffa3146

• Fix an overflow-related failure of chain training, identified by Xiang Li.

  danpovey

  committed
  e2a01c0

• hardening the chain computation a little more against numerical overflow errors

  danpovey

  committed
  2fa82dc

• jesus branch: new example scripts

  danpovey

  committed
  deb5dbe

• jesus branch: adding support for new-style derivative weights

  danpovey

  committed
  d56c870

• fix regarding how return status from denominator computation is handled

  danpovey

  committed
  4c938a8

• Merge pull request kaldi-asr#444 from Timmmm/windows_line_endings

  Show description for 1017986
  jtrmal

  committed
  1017986

• Merge pull request kaldi-asr#445 from Timmmm/windows_docs

  Show description for ad004c4
  jtrmal

  committed
  ad004c4

• Clarify the generate_solution.pl command in the Windows INSTALL

  Show description for 138a9c8
  

  Tim Hutt

  committed
  138a9c8

• Formatting clean up and convert INSTALL to markdown

  

  Tim Hutt

  committed
  98e45e2

• Rename INSTALL to INSTALL.md so github renders it.

  

  Tim Hutt

  committed
  1ee5616

• Github markdown requires more spaces

  Timmmm

  committed
  4456ab0

Commits on Jan 14, 2016

• Adding code to support l2 regularization in training chain models

  danpovey

  committed
  c976566

• Merging changes from jesus branch into jesus-cut-zero branch, RE l2 regularization

  danpovey

  committed
  0dc60fb

• Bug-fixes to previous commit, RE l2 regularization

  danpovey

  committed
  a729d9c

• jesus-cut-zero: merging changes from jesus branch

  danpovey

  committed
  f955354

• jesus branch: merging changes from jesus-cut-zero: example script changes, and script-level support for l2 regularization

  danpovey

  committed
  d3239d0

Commits on Jan 15, 2016

• jesus branch: merging changes from jesus-cut-zero branch... although this method of zeroing the derivatives near cut points was not helpful for TDNNs, it may be helpful for LSTMs, and we can later …

  Show description for 242ec8c
  danpovey

  committed
  242ec8c

• Implement --convert-repeated-to-block option in nnet3-am-copy.

  Show description for 1fce153
  galv

  committed
  1fce153

• jesus branch: merging changes from chain branch

  danpovey

  committed
  c903384

• minor documentation change

  danpovey

  committed
  1698b39

• bug fix to a rather old script: get_lda_block.sh (in fact this reverts the most recent commit).

  danpovey

  committed
  83b94ae

• Style fixes.

  galv

  committed
  7c68d0c

• some script cleanup for jesus-layer related scripts

  danpovey

  committed
  4fa93e0

• removing deprecated jesus-layer-related script

  danpovey

  committed
  f8be514

Commits on Jan 16, 2016

• chain branch: some more recent tuning scripts, testing l2-regularization and smaller jesus-hidden-dim

  danpovey

  committed
  7e0cc88

Commits on Jan 17, 2016

• Merge pull request kaldi-asr#451 from galv/repeatedtoblockaffine

  Show description for 2bdc3a5
  danpovey

  committed
  2bdc3a5

Commits on Jan 18, 2016

• sequenuce: Some clean up to discriminative-training

  vimalmanohar

  committed
  43e7f58

• sequence: Normalizing the initial and final scores added while splitting lattice

  vimalmanohar

  committed
  8a7f43f

• sequence: Modified nnet-discriminative-diagnostics to support more options

  vimalmanohar

  committed
  d73bab8

• sequence: Added nnet3-modify-learning-rates.cc

  vimalmanohar

  committed
  4555567

• sequence: Minor cosmetic change to nnet-discriminative-example.cc

  vimalmanohar

  committed
  f845099

• sequence: Added some testing code to nnet3 discriminative training

  vimalmanohar

  committed
  55ea7c9

Commits on Jan 19, 2016

• Separated ARPA parsing from const LM construction

  

  kkm

  committed
  e0cbd32

• Clarify that MLK and OpenBLAS are alternatives.

  

  Tim Hutt

  committed
  9abd21c

Commits on Jan 21, 2016

• Added PerDimWeightedAverage component for chain models in SWBD, works as good as jesus (tdnn_4f) model

  vijayaditya

  committed
  354d7c2

• Merge pull request kaldi-asr#462 from vijayaditya/chain_feature2

  Show description for 763676a
  danpovey

  committed
  763676a

Commits on Jan 22, 2016

• minor bug correction in steps/nnet3/tdnn/make_configs.py

  vijayaditya

  committed
  a629bc8

• Merge pull request kaldi-asr#463 from vijayaditya/chain_feature2

  Show description for 7878aae
  danpovey

  committed
  7878aae

• Improved error messages for ARPA file parsing

  Show description for 6265183
  

  kkm

  committed
  6265183

• Minor bug removal in tdnn config generator

  vijayaditya

  committed
  a140bdc

• Merge pull request kaldi-asr#465 from vijayaditya/chain_feature2

  Show description for c2129e5
  danpovey

  committed
  c2129e5

• minor bug removal in local/chain/run_tdnn_4q.sh

  vijayaditya

  committed
  981db1e

• Merge pull request kaldi-asr#466 from vijayaditya/chain_feature2

  Show description for 1c56724
  danpovey

  committed
  1c56724

• Changes per @danpovey's review in kaldi-asr#458

  

  kkm

  committed
  1aab0b6

• Merge pull request kaldi-asr#458 from kkm000/arpa-1

  Show description for 61a551d
  danpovey

  committed
  61a551d

Commits on Jan 23, 2016

• More fixes for bugs introduced due to PR kaldi-asr#462

  vijayaditya

  committed
  bd0208f

• Merge pull request kaldi-asr#467 from vijayaditya/chain_feature2

  Show description for 50f2083
  danpovey

  committed
  50f2083

• Merge pull request kaldi-asr#448 from Timmmm/windows_docs

  Show description for 549af84
  jtrmal

  committed
  549af84

• chain branch: adding 4r example script with slightly better jesus-layer configuration.

  danpovey

  committed
  8f2aa7f

• implementing leaky-hmm idea

  danpovey

  committed
  54960bc

• chain-leaky-hmm branch: adding leaky-hmm options to script, and various bug-fixes in code; adding example scripts

  danpovey

  committed
  5ecc1de

Commits on Jan 24, 2016

• Adding code and script options for cross-entropy regularization of chain models

  danpovey

  committed
  4efcde7

• merging chain-xent changes in to leaky-hmm branch for convenient testing while other job is running

  danpovey

  committed
  9ef4a3a

• Various bug-fixes for the chain-xent (cross-entropy smoothing of chain models) to work.

  danpovey

  committed
  e37091e

• chain branch: various tuning scripts for swbd/s5c, demonstrating leaky-hmm (seems slightly helpful with large enough coefficient) and cross-entropy smoothing (definitely helpful, still tuning).

  danpovey

  committed
  6d1da0d

• chain branch: Adding results for 4v (regarding cross-entropy regularization).

  danpovey

  committed
  c3fedd1

Commits on Jan 25, 2016

• Code simplification and cleanup that was enabled by the implementation leaky-hmm (got rid of the 'special state').

  danpovey

  committed
  ffcb552

• chain branch: add sorting on num-transitions (for very tiny speedup).

  danpovey

  committed
  4d42ea2

• chain branch: adding results to a script; a couple of new scripts.

  danpovey

  committed
  ca2772b

Commits on Jan 26, 2016

• chain branch: script edits with new results shown.

  danpovey

  committed
  3342530

• Merge branch 'master' into chain

  danpovey

  committed
  2f585de

• change to arpa-file-parser.cc to suppress spurious compiler warning

  danpovey

  committed
  e46406b

• Fix compilation issues when using KALDI_DOUBLEPRECISION=1 caused by hardcoded types

  funous

  committed
  a422890

Commits on Jan 27, 2016

• chain branch: various new tuning-scripts for chain model.

  danpovey

  committed
  3431b74

• chain branch: more Switchboard tuning scripts, with results.

  danpovey

  committed
  8676636

Commits on Jan 28, 2016

• remove redundant type hints

  funous

  committed
  f587473

• Added some scripts for parsing nnet3 logs

  vijayaditya

  committed
  a125eb4

• chain models: add results from tuning scripts

  danpovey

  committed
  37261b5

• Merge pull request kaldi-asr#471 from vijayaditya/new_feature2

  Show description for e418aa5
  danpovey

  committed
  e418aa5

• Merge pull request kaldi-asr#470 from speechmatics/bugfix-doubleprecision

  Show description for 75a8334
  danpovey

  committed
  75a8334

Commits on Jan 29, 2016

• confidence calibration: the lattice depths in data preparation are now optional

  KarelVesely84

  committed
  2cf78a7

• Copy/paste bug in nnet3/get_egs.sh when using fMLLR

  alumae

  committed
  b515e03

• Merge pull request kaldi-asr#472 from alumae/patch-1

  Show description for 5b98994
  danpovey

  committed
  5b98994

• minor bug-fix in cudamatrix code (forgot to check error status)

  danpovey

  committed
  d165318

• selectively merging from xiaohui-zhang's branch, some code for writing/reading vectors of pairs.

  danpovey

  committed
  eb13407

• Merge remote-tracking branch 'upstream/master' into chain

  danpovey

  committed
  ad7951d

Commits on Jan 31, 2016

• sequence: Minor changes to nnet3 sequence get working code

  vimalmanohar

  committed
  6294bf3

• sequence: Added a version of lattice-determinize that saves the acoustic scores in their original frames

  vimalmanohar

  committed
  9bfb6fa

• sequence: Added test functions and also converting nnet2 to nnet3 degs

  vimalmanohar

  committed
  d3ce34a

• sequence: nnet3 sequeuce scripts

  vimalmanohar

  committed
  e8d0b75

• sequence: Minor fix in nnet3/decode.sh

  vimalmanohar

  committed
  accfc59

• sequence: Fixed bug in nnet2 sequence training that did not take context from the network

  vimalmanohar

  committed
  22e4df2

• sequence: nnet3 sequence scripts in swbd and wsj

  vimalmanohar

  committed
  4630e94

Commits on Feb 1, 2016

• sequence: minor modifications to discriminative nnet2 scripts

  vimalmanohar

  committed
  7d9c43b

• sequence: Bug fix to write frame_subsampling_factor to nnet3 info dir

  vimalmanohar

  committed
  a1e930c

• sequence: Bug fix to scoring scripts

  vimalmanohar

  committed
  7da9a8c

• snr: Added length tolerance to nnet3-get-egs

  vimalmanohar

  committed
  1bcb8e1

Commits on Feb 3, 2016

• chain branch: minor bugfix (remove duplicately registered option)

  danpovey

  committed
  b3f05c8

• sequence: Minor fixes

  vimalmanohar

  committed
  9280600

• sequence: Fixed sequence script

  vimalmanohar

  committed
  55ba492

• sequence : Added some options to top level scripts

  vimalmanohar

  committed
  506c3e3

• sequence: restoring old behavior of old behavior of smbr for now

  vimalmanohar

  committed
  b7dea22

• sequence: Fixed some bugs in nnet3/nnet-utils.cc

  vimalmanohar

  committed
  9bf0acf

Commits on Feb 12, 2016

• snr: Added weights option to online2bin/ivector-extract-online2.cc

  vimalmanohar

  committed
  3f20fa6

Commits on Feb 14, 2016

• sequence: Fixed bug to directly read from normal Lattice type and added regularization

  vimalmanohar

  committed
  34f2ffe

• sequence: Made a bunch of script modifications to support chain models with discriminative training

  vimalmanohar

  committed
  b5c2412

• sequence: Moved around some functions common to chain and discriminative training

  vimalmanohar

  committed
  e2e676a

• sequeunce: Support reading subset in lattice-copy

  vimalmanohar

  committed
  707ea5e

• sequence: swbd discriminative scripts top level

  vimalmanohar

  committed
  fc62a1c

• sequence: Added python explicitly for calling python scripts

  vimalmanohar

  committed
  502d25e

• sequence: nnet3 adjust priors script

  vimalmanohar

  committed
  25acfaa

• semisup: Modified scoring script to get confidences

  vimalmanohar

  committed
  05b21dc

• sequence: Added some programs to work with degs

  vimalmanohar

  committed
  3d2129a