论文标题
实时手势动画从虚拟人类互动的语音中产生
Real-time Gesture Animation Generation from Speech for Virtual Human Interaction
论文作者
论文摘要
我们建议直接从语音中综合手势的实时系统。我们的数据驱动方法基于生成的对抗性神经网络，以模拟语音传语的关系。我们利用在线可用的大量扬声器视频数据来训练我们的3D手势模型。我们的模型通过连续两秒钟的连续音频输入块来生成特定于扬声器的手势。我们在虚拟化身上对预测的手势进行动画动画。在音频输入和手势动画的时间之间，我们达到了低于三秒钟的延迟。代码和视频可从https://github.com/mrebol/gestures-from-speech获得
We propose a real-time system for synthesizing gestures directly from speech. Our data-driven approach is based on Generative Adversarial Neural Networks to model the speech-gesture relationship. We utilize the large amount of speaker video data available online to train our 3D gesture model. Our model generates speaker-specific gestures by taking consecutive audio input chunks of two seconds in length. We animate the predicted gestures on a virtual avatar. We achieve a delay below three seconds between the time of audio input and gesture animation. Code and videos are available at https://github.com/mrebol/Gestures-From-Speech
