A Bayesian Interpretation of Interpolated Kneser-Ney
No Thumbnail Available
Files
Date
2006-02-02T09:23:28Z
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Interpolated Kneser-Ney is one of the best smoothing methods for n-gram language models. Previous explanations for its superiority have been based on intuitive and empirical justifications of specific properties of the method. We propose a novel interpretation of
interpolated Kneser-Ney as approximate inference in a hierarchical Bayesian model consisting of Pitman-Yor processes. As opposed to past explanations, our interpretation can recover exactly the formulation of interpolated Kneser-Ney, and performs better than interpolated Kneser-Ney when a better inference procedure is used.