论文标题
Unigram规范化的困惑作为一种语言模型性能度量，具有不同的词汇大小
Unigram-Normalized Perplexity as a Language Model Performance Measure with Different Vocabulary Sizes
论文作者
论文摘要
尽管困惑性是语言模型的广泛使用的性能度量，但这些值高度依赖于语料库中的单词数量，并且对于仅比较同一语料库的性能很有用。在本文中，我们提出了一个新的指标，可用于评估不同词汇大小的语言模型性能。拟议的Unigram规范化的困惑实际上呈现了简单的Unigram模型的语言模型的性能提高，并且对词汇量的大小非常强大。均报道了理论分析和计算实验。
Although Perplexity is a widely used performance metric for language models, the values are highly dependent upon the number of words in the corpus and is useful to compare performance of the same corpus only. In this paper, we propose a new metric that can be used to evaluate language model performance with different vocabulary sizes. The proposed unigram-normalized Perplexity actually presents the performance improvement of the language models from that of simple unigram model, and is robust on the vocabulary size. Both theoretical analysis and computational experiments are reported.
