论文标题
通过情节知识图评估代理相互作用
Evaluating Agent Interactions Through Episodic Knowledge Graphs
论文作者
论文摘要
我们提出了一种基于情节知识图（EKG）的新方法，用于评估开放域中的（多模式）对话剂。该图是通过解释对话过程中的原始信号而生成的，并且能够随着时间的推移捕获知识的积累。我们应用对所得图的结构和语义分析，并将这些属性转化为定性措施。我们将这些措施与通常用于会话代理的现有自动和手动评估指标进行比较。我们的结果表明，我们的基于知识的评估为互动和代理人的行为提供了更多的定性见解。
We present a new method based on episodic Knowledge Graphs (eKGs) for evaluating (multimodal) conversational agents in open domains. This graph is generated by interpreting raw signals during conversation and is able to capture the accumulation of knowledge over time. We apply structural and semantic analysis of the resulting graphs and translate the properties into qualitative measures. We compare these measures with existing automatic and manual evaluation metrics commonly used for conversational agents. Our results show that our Knowledge-Graph-based evaluation provides more qualitative insights into interaction and the agent's behavior.
