论文标题
在口语理解系统中的假设拒绝模块的设计注意事项
Design Considerations For Hypothesis Rejection Modules In Spoken Language Understanding Systems
论文作者
论文摘要
口语理解（SLU）系统通常由一组机器学习模型组成，这些模型结合使用以产生SLU假设。然后将生成的假设发送到下游组成部分，以采取进一步的行动。但是，希望在下游发送之前放弃错误的假设。在这项工作中，我们介绍了SLU假设排斥模块的两种设计：（i）对拒绝域特定SLU假设的拒绝和（II）方案R2的方案R1，该方案R2对从整体SLU系统产生的假设进行排斥。两种方案中的假设排斥模块拒绝/接受基于从针对SLU系统的话语（相关SLU假设和SLU置信度得分）的特征的特征。我们的实验表明，这两个方案都产生相似的结果（方案R1：2.5％FRR @ 4.5％远，方案R2：2.5％FRR @ 4.6％远4.6％），使用所有可用功能的性能最佳。我们认为，尽管可以选择任何一种拒绝方案，但它们具有一些固有的差异，在做出此选择时需要考虑这些差异。此外，我们将ASR功能纳入拒绝模块（获得1.9％的FRR @ 3.8％）并分析改进。
Spoken Language Understanding (SLU) systems typically consist of a set of machine learning models that operate in conjunction to produce an SLU hypothesis. The generated hypothesis is then sent to downstream components for further action. However, it is desirable to discard an incorrect hypothesis before sending it downstream. In this work, we present two designs for SLU hypothesis rejection modules: (i) scheme R1 that performs rejection on domain specific SLU hypothesis and, (ii) scheme R2 that performs rejection on hypothesis generated from the overall SLU system. Hypothesis rejection modules in both schemes reject/accept a hypothesis based on features drawn from the utterance directed to the SLU system, the associated SLU hypothesis and SLU confidence score. Our experiments suggest that both the schemes yield similar results (scheme R1: 2.5% FRR @ 4.5% FAR, scheme R2: 2.5% FRR @ 4.6% FAR), with the best performing systems using all the available features. We argue that while either of the rejection schemes can be chosen over the other, they carry some inherent differences which need to be considered while making this choice. Additionally, we incorporate ASR features in the rejection module (obtaining an 1.9% FRR @ 3.8% FAR) and analyze the improvements.
