论文标题
在边缘云环境中对分布式数据处理平台评估的能源感知评估框架
A Framework for Energy-aware Evaluation of Distributed Data Processing Platforms in Edge-Cloud Environment
论文作者
论文摘要
分布式数据处理平台（例如，Hadoop，Spark和Flink）广泛用于在云的计算节点之间分配数据的存储和处理。云资源的集中化已经诞生了边缘计算，这使数据可以更接近数据源的处理，而不是将其发送到云中。但是，由于资源限制（例如能量限制），Edge计算不能用于部署各种应用程序。因此，将任务从边缘设备卸载到更具机智的云。先前的研究评估了在孤立的云和边缘环境中分布式数据处理平台的能耗。但是，在集成的边缘环境中评估这些平台的能源消耗的研究很少，在该环境中，任务从资源构成设备转移到资源丰富的设备。因此，在本文中，我们首先介绍了分布式数据处理平台的能源感知评估的框架。然后，我们利用提出的框架来评估由Raspberry Pi，Edge Node，Edge Server节点，Private Cloud和Public Cloud组成的集成边缘云环境中三个最广泛使用的平台（即Hadoop，Spark和Flink）的能耗。我们的评估表明，（i）Flink是最能节能的，其次是火花和Hadoop，发现从资源控制到资源丰富的设备的卸载任务最少（II）将能源消耗降低了55.2％，并且（iii）客户和服务器之间的带宽和距离是对能源消耗的关键因素。
Distributed data processing platforms (e.g., Hadoop, Spark, and Flink) are widely used to distribute the storage and processing of data among computing nodes of a cloud. The centralization of cloud resources has given birth to edge computing, which enables the processing of data closer to the data source instead of sending it to the cloud. However, due to resource constraints such as energy limitations, edge computing cannot be used for deploying all kinds of applications. Therefore, tasks are offloaded from an edge device to the more resourceful cloud. Previous research has evaluated the energy consumption of the distributed data processing platforms in the isolated cloud and edge environments. However, there is a paucity of research on evaluating the energy consumption of these platforms in an integrated edge-cloud environment, where tasks are offloaded from a resource-constraint device to a resource-rich device. Therefore, in this paper, we first present a framework for the energy-aware evaluation of the distributed data processing platforms. We then leverage the proposed framework to evaluate the energy consumption of the three most widely used platforms (i.e., Hadoop, Spark, and Flink) in an integrated edge-cloud environment consisting of Raspberry Pi, edge node, edge server node, private cloud, and public cloud. Our evaluation reveals that (i) Flink is most energy-efficient followed by Spark and Hadoop is found least energy-efficient (ii) offloading tasks from resource-constraint to resource-rich devices reduces energy consumption by 55.2%, and (iii) bandwidth and distance between client and server are found key factors impacting the energy consumption.
