Exploring SIMD Vectorization in Aggregation Pipelines for Encoded IoT Data

Time-series databases have been critical for collecting and analyzing data in industries where sensors send large amounts of IoT data by network devices. Both data received from networks and data collected in database storage are sufficiently encoded to reduce I/O occupation and latency. The IoT encoders successively combine the Delta, Repeat, and Packing operators, yielding a higher compression ratio than simply adopting each. However, efficient compression makes query execution even harder, requiring serial decoding before processing queries. Among them, selective aggregations, such as down-sampling, are the core of time series analytical queries. This paper identifies operators to process and accelerate IoT aggregation queries based on encoded data arrays, extensible to integrate thread-level and instruction-level designs. In addition, encoded data could aggregate directly in parallel without decoding, and encoding statistics can help to reduce unnecessary computation. Identified operators construct a pipeline query engine to integrate into an existing open source database, the Apache IoTDB. Remarkably, our systemic evaluations show vast improvements in the efficiency of selective aggregation over existing works.