Improving GPU Tensor Query Processing for Resource-Constrained Environments

Over the past decade, artificial intelligence (AI) has made remarkable strides, driven by researchers, engineers, and vendors who have relentlessly developed high-performance libraries across heterogeneous hardware and software. Recently, researchers in the database (DB) community have begun to harness this progress by representing data as tensors and redefining large-scale data operations through tensor-based computations. This approach seeks to leverage the rapid advancements in AI technologies. Typically, AI computing devices are heterogeneous, with limited and fixed on-device memory. We identify the lack of ability to process large-scale data as the biggest challenge in bridging the gap between the worlds of AI and databases. We propose TensorSlim, a memory-efficient big data analytical system incorporating three key innovations. First, we build a highly optimized operator execution engine that keeps GPU usage bounded by reusing buffers and releasing intermediates early. Second, we propose a tensor-driven processing framework that treats peak GPU memory as a first-class constraint and performs operator-aware GPU memory budgeting to guide query partitioning. Third, we develop a dependency-aware hybrid CPU–GPU query optimizer to manage dataflow dependencies and device placements, avoiding large GPU-resident intermediates. We demonstrate that TensorSlim can deliver 25.5× faster query speeds than traditional GPU-based solutions while achieving 6.4× speedup over CPU-based solutions on average. Additionally, TensorSlim reduces peak memory usage by 73.5%.