论文标题
BOTSIM：针对商业任务对话框系统的端到端BOT模拟框架
BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems
论文作者
论文摘要
我们提供BOTSIM，这是一种用于基于文本的任务对话框（TOD）系统的数据效率端到端BOT模拟工具包。 BOTSIM由三个主要组成部分组成：1）可以从机器人定义中推断出语义级对话框和实体的生成器，并通过基于模型的释义生成用户查询； 2）基于议程的对话框用户模拟器（ABU），以模拟与对话框的对话； 3）分析模拟对话，可视化机器人健康报告并为机器人故障排除和改进提供可行的补救建议的补救措施。我们证明了BOTSIM在两个商业机器人平台上的案例研究中的端到端评估，补救和多面对话的有效性。博茨的“生成模拟解析”范式通过以下方式加速了端到端的机器人评估和迭代过程：1）减少手动测试案例的创建工作； 2）通过广泛的对话框模拟，可以根据NLU和端到端的性能启用机器人的整体规格； 3）使用可操作的建议改善机器人故障排除过程。可以在https://tinyurl.com/mryu74cd和https://youtu.be/qli5isoly30上找到我们系统的演示。我们已通过https://github.com/salesforce/botsim开源该工具包
We present BotSIM, a data-efficient end-to-end Bot SIMulation toolkit for commercial text-based task-oriented dialog (TOD) systems. BotSIM consists of three major components: 1) a Generator that can infer semantic-level dialog acts and entities from bot definitions and generate user queries via model-based paraphrasing; 2) an agenda-based dialog user Simulator (ABUS) to simulate conversations with the dialog agents; 3) a Remediator to analyze the simulated conversations, visualize the bot health reports and provide actionable remediation suggestions for bot troubleshooting and improvement. We demonstrate BotSIM's effectiveness in end-to-end evaluation, remediation and multi-intent dialog generation via case studies on two commercial bot platforms. BotSIM's "generation-simulation-remediation" paradigm accelerates the end-to-end bot evaluation and iteration process by: 1) reducing manual test cases creation efforts; 2) enabling a holistic gauge of the bot in terms of NLU and end-to-end performance via extensive dialog simulation; 3) improving the bot troubleshooting process with actionable suggestions. A demo of our system can be found at https://tinyurl.com/mryu74cd and a demo video at https://youtu.be/qLi5iSoly30. We have open-sourced the toolkit at https://github.com/salesforce/botsim
