| Paper: | MLSP-L2.4 | ||
| Session: | Kernel Machines | ||
| Time: | Wednesday, May 17, 17:30 - 17:50 | ||
| Presentation: | Lecture | ||
| Topic: | Machine Learning for Signal Processing: Graphical and kernel models | ||
| Title: | Sparse Forward-Backward using Minimum Divergence Beams for Fast Training of Conditional Random Fields | ||
| Authors: | Chris Pal, Charles Sutton, Andrew McCallum, University of Massachusetts, United States | ||
| Abstract: | Hidden Markov models and linear-chain conditional random fields (CRFs) are applicable to many tasks in spoken language processing. In large state spaces, however, training can be expensive, because it often requires many iterations of forward-backward. Beam search is a standard heuristic for controlling complexity during Viterbi decoding, but during forward-backward, standard beam heuristics can be dangerous, as they can make training unstable. We introduce sparse forward-backward, a variational perspective on beam methods that uses an approximating mixture of Kronecker delta functions. This motivates a novel minimum-divergence beam criterion based on minimizing KL divergence between the respective marginal distributions. Our beam selection approach is not only more efficient for Viterbi decoding, but also more stable within sparse forward-backward training. For a standard text-to-speech problem, we reduce CRF training time fourfold---from over a day to six hours---with no loss in accuracy. | ||
©2018 Conference Management Services, Inc. -||- email: webmaster@icassp2006.org -||- Last updated Friday, August 17, 2012