论文标题

GX-PLUG:用于插入加速器以分布式图形处理的中间件

GX-Plug: a Middleware for Plugging Accelerators to Distributed Graph Processing

论文作者

Zou, Kai, Xie, Xike, Li, Qi, Kong, Deyu

论文摘要

最近,研究社区强调了为大型图形处理制定可扩展性连续性的必要性,该处理能够从分布式图系统中获得扩展利益,而扩大范围则受益于高性能加速器。为此,我们提出了一种称为GX-Plug的中间件,以便于两者的优点。作为中间件,GX插头在支持不同的运行时环境,计算模型和编程模型方面具有多功能性。更重要的是,为了提高中间件性能,我们研究了一系列技术,包括管道洗牌,同步缓存和跳过,以及工作负载平衡,分别用于内部,间和超越idirate的优化​​。实验表明,我们的中间件有效地将加速器插入了代表性的分布式图系统,例如GraphX和PowerGraph,具有最高的加速度比率。

Recently, research communities highlight the necessity of formulating a scalability continuum for large-scale graph processing, which gains the scale-out benefits from distributed graph systems, and the scale-up benefits from high-performance accelerators. To this end, we propose a middleware, called the GX-plug, for the ease of integrating the merits of both. As a middleware, the GX-plug is versatile in supporting different runtime environments, computation models, and programming models. More, for improving the middleware performance, we study a series of techniques, including pipeline shuffle, synchronization caching and skipping, and workload balancing, for intra-, inter-, and beyond-iteration optimizations, respectively. Experiments show that our middleware efficiently plugs accelerators to representative distributed graph systems, e.g., GraphX and Powergraph, with up-to 20x acceleration ratio.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源