论文标题
通过听众自适应跨域融合识别多模式二二元印象
Multimodal Dyadic Impression Recognition via Listener Adaptive Cross-Domain Fusion
论文作者
论文摘要
作为情感计算的子分支，印象识别，例如对诸如温暖或能力之类的说话者特征的看法，可能是人类对话和口头对话系统的关键部分。大多数研究仅从说话者表达的行为或听众的反应中研究了印象，但忽略了他们的潜在联系。在本文中，我们使用提出的听众自适应跨域体系结构执行印象识别，该体系结构由侦听器的自适应功能组成，以模拟说话者和听众行为之间的因果关系以及跨域融合函数，以增强其联系。对二元印象数据集的实验评估验证了我们方法的疗效，在能力和温暖维度中产生了一致性相关系数78.8％和77.5％，超过了先前的研究。预计所提出的方法将被推广到类似的二元相互作用方案。
As a sub-branch of affective computing, impression recognition, e.g., perception of speaker characteristics such as warmth or competence, is potentially a critical part of both human-human conversations and spoken dialogue systems. Most research has studied impressions only from the behaviors expressed by the speaker or the response from the listener, yet ignored their latent connection. In this paper, we perform impression recognition using a proposed listener adaptive cross-domain architecture, which consists of a listener adaptation function to model the causality between speaker and listener behaviors and a cross-domain fusion function to strengthen their connection. The experimental evaluation on the dyadic IMPRESSION dataset verified the efficacy of our method, producing concordance correlation coefficients of 78.8% and 77.5% in the competence and warmth dimensions, outperforming previous studies. The proposed method is expected to be generalized to similar dyadic interaction scenarios.
