论文标题
通过手势动作分类和自回归模型从文本中产生3D人类运动
3d human motion generation from the text via gesture action classification and the autoregressive model
论文作者
论文摘要
在本文中，通过手势动作分类和自回归模型提出了从文本中产生3D人类运动的深度学习模型。该模型着重于产生表达人类思维的特殊手势，例如挥舞和点头。为了实现目标，提出的方法使用基于验证的语言模型的文本分类模型从句子中预测表达，并使用基于门的基于单元的自动回应模型生成手势。尤其是，我们提出了嵌入空间的损失，以恢复原始动作并很好地产生中间运动。此外，提出了新型的数据增强方法和停止令牌来产生可变长度运动。为了评估文本分类模型和3D人类运动生成模型，收集了手势动作分类数据集和基于动作的手势数据集。通过几个实验，提出的方法成功地从文本中成功产生了自然和现实的3D人类运动。此外，我们使用公共可用的动作识别数据集验证了所提出的方法的有效性，以评估跨数据集泛化性能。
In this paper, a deep learning-based model for 3D human motion generation from the text is proposed via gesture action classification and an autoregressive model. The model focuses on generating special gestures that express human thinking, such as waving and nodding. To achieve the goal, the proposed method predicts expression from the sentences using a text classification model based on a pretrained language model and generates gestures using the gate recurrent unit-based autoregressive model. Especially, we proposed the loss for the embedding space for restoring raw motions and generating intermediate motions well. Moreover, the novel data augmentation method and stop token are proposed to generate variable length motions. To evaluate the text classification model and 3D human motion generation model, a gesture action classification dataset and action-based gesture dataset are collected. With several experiments, the proposed method successfully generates perceptually natural and realistic 3D human motion from the text. Moreover, we verified the effectiveness of the proposed method using a public-available action recognition dataset to evaluate cross-dataset generalization performance.
