论文标题
SDN帮助大数据优化对数据的访问
SDN helps Big Data to optimize access to data
论文作者
论文摘要
本章介绍了将高性能计算（HPC）与大数据分析相结合的新兴领域的最新领域。为了了解新领域，本章首先调查将HPC与大数据集成的现有方法。接下来，本章介绍了几种优化解决方案，这些解决方案的重点是如何最大程度地减少数据传输时间从计算密集型应用程序到分析密集型应用程序，并最大程度地减少了端到端的时间到达时间。该解决方案利用SDN适应使用高速互连网络和高性能并行文件系统来优化应用程序性能。设计和开发了一个称为DataBroker的计算框架，以通过数据分析实现HPC的紧密整合。已经进行了多种类型的实验，以在消息传递和并行文件系统中显示出不同的性能问题，并验证拟议的研究方法的有效性。
This chapter introduces the state-of-the-art in the emerging area of combining High Performance Computing (HPC) with Big Data Analysis. To understand the new area, the chapter first surveys the existing approaches to integrating HPC with Big Data. Next, the chapter introduces several optimization solutions that focus on how to minimize the data transfer time from computation-intensive applications to analysis-intensive applications as well as minimizing the end-to-end time-to-solution. The solutions utilize SDN to adaptively use both high speed interconnect network and high performance parallel file systems to optimize the application performance. A computational framework called DataBroker is designed and developed to enable a tight integration of HPC with data analysis. Multiple types of experiments have been conducted to show different performance issues in both message passing and parallel file systems and to verify the effectiveness of the proposed research approaches.
