Today's scientific simulations require a significant reduction of data volume because of extremely large amounts of data they produce and the limited I/O bandwidth and storage space. Error-bounded lossy compression has been considered one of the most effective solutions to the above problem. However, little work has been done to improve error-bounded lossy compression for Adaptive Mesh Refinement (AMR) simulation data. Unlike the previous work that only leverages 1D compression, in this work, we propose to leverage high-dimensional (e.g., 3D) compression for each refinement level of AMR data. To remove the data redundancy across different levels, we propose three pre-process strategies and adaptively use them based on the data characteristics. Experiments on seven AMR datasets from a real-world large-scale AMR simulation demonstrate that our proposed approach can improve the compression ratio by up to 3.3X under the same data distortion, compared to the state-of-the-art method. In addition, we leverage the flexibility of our approach to tune the error bound for each level, which achieves much lower data distortion on two application-specific metrics.
翻译:今天的科学模拟需要大量减少数据量,因为其产生的数据数量巨大,而且I/O带宽和储存空间有限。 错误造成的损耗压缩被认为是解决上述问题的最有效办法之一。 但是,在改进适应性网状精炼(AMR)模拟数据中误差引起的损耗压缩方面,没有做多少工作。 与以前只利用1D压缩的工作不同,在这项工作中,我们提议利用高维(例如3D)压缩法对AMR数据的每个精细水平进行调整。 为了消除不同层次的数据冗余,我们提议了三个预处理战略,并根据数据特性适应性地使用这些战略。 从一个真实的大规模AMR模拟中进行的七个AMR数据集实验表明,我们所提议的方法可以在同一数据扭曲下,将压缩率提高到3.3X,而采用最新的方法。 此外,我们利用我们的方法的灵活性来调整每个层次的误差,从而在两个应用特定的计量标准上实现更低得多的数据扭曲。