论文标题
肌肉在作用
Muscles in Action
论文作者
论文摘要
人类运动是由我们的肌肉创造的，并由我们的肌肉束缚。我们迈出了代表导致运动的内部肌肉活动的计算机视觉方法的第一步。我们提出了一个新的数据集，即动作中的肌肉（MIA），以学习将肌肉活动纳入人类运动表示。该数据集由10个受试者进行各种练习的12.5小时同步视频和表面肌电图（SEMG）数据组成。使用此数据集，我们学习了一个双向表示，该表示可以预测视频中的肌肉激活，相反，从肌肉激活中重建运动。我们评估我们的模型，以及分布主题和练习，以及分布的主题和练习。我们证明了如何共同建模两种模式的进步可以作为肌肉一致运动产生的条件。将肌肉放入计算机视觉系统中，可以通过运动，健身和AR/VR的应用来实现更丰富的虚拟人类模型。
Human motion is created by, and constrained by, our muscles. We take a first step at building computer vision methods that represent the internal muscle activity that causes motion. We present a new dataset, Muscles in Action (MIA), to learn to incorporate muscle activity into human motion representations. The dataset consists of 12.5 hours of synchronized video and surface electromyography (sEMG) data of 10 subjects performing various exercises. Using this dataset, we learn a bidirectional representation that predicts muscle activation from video, and conversely, reconstructs motion from muscle activation. We evaluate our model on in-distribution subjects and exercises, as well as on out-of-distribution subjects and exercises. We demonstrate how advances in modeling both modalities jointly can serve as conditioning for muscularly consistent motion generation. Putting muscles into computer vision systems will enable richer models of virtual humans, with applications in sports, fitness, and AR/VR.
