We present a record-breaking result and lessons learned in practicing TPCx-IoT benchmarking for a real-world use case. We find that more system characteristics need to be benchmarked for its application to real-world use cases. We introduce an extension to the TPCx-IoT benchmark, covering fundamental requirements of time-series data management for IoT infrastructure. We characterize them as data compression and system scalability. To evaluate these two important features of IoT databases, we propose IoTDataBench and update four aspects of TPCx-IoT, i.e., data generation, workloads, metrics and test procedures. Preliminary evaluation results show systems that fail to effectively compress data or flexibly scale can negatively affect the redesigned metrics, while systems with high compression ratios and linear scalability are rewarded in the final metrics. Such systems have the ability to scale up computing resources on demand and can thus save dollar costs.
翻译:我们提出了一个破纪录的结果,并总结了在实际使用案例方面采用TPCx-IoT基准所取得的教训。我们发现,需要为将更多的系统特性应用于现实使用案例制定基准。我们引入了TPCx-IoT基准的扩展,涵盖IoT基础设施的时间序列数据管理的基本要求。我们将其定性为数据压缩和系统可缩放性。为了评估IoT数据库的这两个重要特征,我们提议IoTDataBench, 并更新TPCx-IoT的四个方面, 即数据生成、工作量、衡量尺度和测试程序。初步评价结果显示,未能有效压缩数据或灵活规模的系统可能对重新设计的衡量标准产生消极影响,而高压缩率和线性缩缩缩率的系统在最终的度量中得到了回报。这些系统有能力根据需求扩大计算资源,从而节省美元成本。