论文标题
在大型内存商店中实现高通量和弹性
Achieving High Throughput and Elasticity in a Larger-than-Memory Store
论文作者
论文摘要
现在，数以百万计的传感器，移动应用程序和机器会产生数十亿个事件。如果在同一台机器上生成事件，则专门的多核钥匙值商店（KVSS）可以以高速率（一台机器上的100 MOP/s）摄入和索引这些事件；但是，要实用和成本效益，他们必须在网络上摄取事件，并在云中遍布云资源。 我们提出ShadowFax，这是一种基于更快的新分布式KVS，它透明地跨越了DRAM，SSD和Cloud Blob存储，同时使用常规的Linux TCP在商品Azure VM上提供130 MOPS/S/VM。除了高单个VM性能之外，ShadowFax还使用了独特的方法来分布重新配置，避免了在正常操作和迁移过程中进行任何服务器端密钥所有权检查或交叉核心协调。因此，ShadowFax可以在17 s中移动负载，以将系统吞吐量提高10 mops/s，而损失很小。与最新的艺术品相比，它具有8倍的吞吐量（比Seastar+Memcached）更好，并且避免了昂贵的I/O在迁移过程中移动冷数据。在12台机器上，ShadowFax保留了其高吞吐量，以执行930 MOPS/S，据我们所知，这是用于大规模数据摄入和索引的分布式KVS的最高报告的吞吐量。
Millions of sensors, mobile applications and machines now generate billions of events. Specialized many-core key-value stores (KVSs) can ingest and index these events at high rates (over 100 Mops/s on one machine) if events are generated on the same machine; however, to be practical and cost-effective they must ingest events over the network and scale across cloud resources elastically. We present Shadowfax, a new distributed KVS based on FASTER, that transparently spans DRAM, SSDs, and cloud blob storage while serving 130 Mops/s/VM over commodity Azure VMs using conventional Linux TCP. Beyond high single-VM performance, Shadowfax uses a unique approach to distributed reconfiguration that avoids any server-side key ownership checks or cross-core coordination both during normal operation and migration. Hence, Shadowfax can shift load in 17 s to improve system throughput by 10 Mops/s with little disruption. Compared to the state-of-the-art, it has 8x better throughput (than Seastar+memcached) and avoids costly I/O to move cold data during migration. On 12 machines, Shadowfax retains its high throughput to perform 930 Mops/s, which, to the best of our knowledge, is the highest reported throughput for a distributed KVS used for large-scale data ingestion and indexing.
