Lune

PPoPP2020Top-tier venue

waveSZ: a hardware-algorithm co-design of efficient lossy compression for scientific data

Jiannan Tian, Sheng Di, Chengming Zhang, Xin Liang, Sian Jin, Dazhao Cheng, Dingwen Tao, Franck Cappello

Abstract

Error-bounded lossy compression is critical to the success of extreme-scale scientific research because of ever-increasing volumes of data produced by today's high-performance computing (HPC) applications. Not only can error-controlled lossy compressors significantly reduce the I/O and storage burden but they can retain high data fidelity for post analysis. Existing state-of-the-art lossy compressors, however, generally suffer from relatively low compression and decompression throughput (up to hundreds of megabytes per second on a single CPU core), which considerably restrict the adoption of lossy compression by many HPC applications especially those with a fairly high data production rate. In this paper, we propose a highly efficient lossy compression approach based on field programmable gate arrays (FPGAs) under the state-of-the-art lossy compression model SZ. Our contributions are fourfold. (1) We adopt a wavefront memory layout to alleviate the data dependency during the prediction for higher-dimensional predictors, such as the Lorenzo predictor. (2) We propose a co-design framework named waveSZ based on the wavefront memory layout and the characteristics of SZ algorithm and carefully implement it by using high-level synthesis. (3) We propose a hardware-algorithm co-optimization method to improve the performance. (4) We evaluate our proposed waveSZ on three real-world HPC simulation datasets from the Scientific Data Reduction Benchmarks and compare it with other state-of-the-art methods on both CPUs and FPGAs. Experiments show that our waveSZ can improve SZ's compression throughput by 6.9X 8.7X over the production version running on a state-of-the-art CPU and improve the compression ratio and throughput by 2.1X and 5.8X on average, respectively, compared with the state-of-the-art FPGA design.

Ask Lune

Know where this paper stands in minutes, not an afternoon.

Lune can read the top-tier papers that cite it and the ones closest to it, and tells you what it claims, what came after and where it falls short, each point cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

its abstract · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. CereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2Shihui Song, Yafan Huang, Peng Jiang, Xiaodong Yu et al.HPDC 2024 · 14 citations
  2. Parallelizing Stream Compression for IoT Applications on Asymmetric MulticoresXianzhi Zeng, Shuhao ZhangICDE 2023 · 10 citations

Related papers

  1. Significantly Improving Lossy Compression for HPC Datasets with Second-Order Prediction and Parameter OptimizationKai Zhao, Sheng Di, Xin Liang, Sihuan Li et al.HPDC 2020 · 77 citations
  2. Ultrafast Error-bounded Lossy Compression for Scientific DatasetsXiaodong Yu, Sheng Di, Kai Zhao, Jiannan Tian et al.HPDC 2022 · 41 citations
  3. Dynamic Quality Metric Oriented Error Bounded Lossy Compression for Scientific DatasetsJinyang Liu, Sheng Di, Kai Zhao, Xin Liang et al.SC 2022 · 33 citations
  4. cuSZ-i: High-Ratio Scientific Lossy Compression on GPUs with Optimized Multi-Level InterpolationJinyang Liu, Jiannan Tian, Shixun Wu, Sheng Di et al.SC 2024 · 17 citations
  5. Boosting Scientific Error-Bounded Lossy Compression through Optimized Synergistic Lossy-Lossless OrchestrationShixun Wu, Jinwen Pan, Jinyang Liu, Jiannan Tian et al.SC 2025 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines