论文标题
全面的图像字幕通过场景图分解
Comprehensive Image Captioning via Scene Graph Decomposition
论文作者
论文摘要
我们通过重新访问图像场景图的表示，解决图像字幕的具有挑战性的问题。我们方法的核心是场景图的分解成一组子图，每个子图捕获了输入图像的语义成分。我们设计了一个深层模型来选择重要的子图，并将每个选定的子图解码为一个目标句子。通过使用子图，我们的模型可以参与图像的不同组件。因此，我们的方法是准确，多样，扎根和可控制的字幕。我们提出了广泛的实验，以证明我们全面字幕模型的好处。我们的方法建立了新的最先进的结果，以标题多样性，接地和可控性，并与标题质量的最新方法进行了比较。我们的项目网站可以在http://pages.cs.wisc.edu/~yiiwuzhong/sub-gc.html上找到。
We address the challenging problem of image captioning by revisiting the representation of image scene graph. At the core of our method lies the decomposition of a scene graph into a set of sub-graphs, with each sub-graph capturing a semantic component of the input image. We design a deep model to select important sub-graphs, and to decode each selected sub-graph into a single target sentence. By using sub-graphs, our model is able to attend to different components of the image. Our method thus accounts for accurate, diverse, grounded and controllable captioning at the same time. We present extensive experiments to demonstrate the benefits of our comprehensive captioning model. Our method establishes new state-of-the-art results in caption diversity, grounding, and controllability, and compares favourably to latest methods in caption quality. Our project website can be found at http://pages.cs.wisc.edu/~yiwuzhong/Sub-GC.html.
