Serf: Streaming Error-Bounded Floating-Point Compression

In IoT (Internet of Things) scenarios, massive floating-point time series data are generated in a streaming manner and transmitted within limited bandwidth for real-time analysis. To enhance the efficiency, it is acknowledged to compress the data before transmission. Existing floating-point compression methods are either for batched compression that may cause long delays, or for streaming lossless compression that has an unsatisfactory compression ratio when certain errors are allowed. In this paper, we propose the first <u>S</u>treaming <u>ER</u>ror-bounded <u>F</u>loating-point compression Serf , which has two implementations: Serf-Qt and Serf-XOR . Serf-Qt first quantizes each floating-point value into an integer, and then encodes the integer with Elias gamma coding. Serf-XOR is the first lossy floating-point compression based on the XORing operation. To enhance the compression ratio of Serf-XOR , we propose a novel data offset technique to increase the leading zeros of the XORed values, and design a novel approximation technique to search for an error-qualified value that produces an XORed value with many trailing zeros. To improve the compression efficiency, we propose a pruning strategy to accelerate the process of approximated values search. We further build a streaming transmission prototype system based on a real development board, and deploy the proposed methods to it. Extensive experiments using 13 datasets show that, compared with 17 competitors, both Serf-Qt and Serf-XOR enjoy remarkable compression ratios with high efficiency in streaming scenarios. The transmission experiments based on the proposed system also showcase that Serf-XOR always takes the least overall time when the bandwidth is limited.