SEMA: A High-performance System for LLM-based Semantic Query Processing

The integration of Large Language Models (LLMs) into data analytics has unlocked powerful capabilities for reasoning over bulk structured and unstructured data. However, existing systems typically rely on either DataFrame primitives, which lack the efficient execution infrastructure of modern DBMSs, or SQL User-Defined Functions (UDFs), which isolate semantic logic from the query optimizer. The LLM-powered semantic operators also bring new challenges due to the high cost and non-deterministic nature of the LLM invocation, where conventional optimization rules and cost models are inapplicable for their optimization. To bridge these gaps, we present Sema, a high-performance semantic query engine built on DuckDB that treats LLM-powered semantic operators as first-class citizens. Sema introduces SemaSQL, a declarative dialect that allows users to seamlessly inject natural language expressions into standard SQL clauses, enabling end-to-end optimization and execution. At the logical level, the optimizer of Sema compresses natural language expressions and deduces relational constraints from semantic operators. At runtime, Sema employs Adaptive Query Execution (AQE) to dynamically reorder operators, fuse semantic operations, and apply prompt batching. This approach seeks a Pareto-optimal execution path that minimizes either token consumption or latency according to user preference, subject to accuracy constraints. We evaluate Sema on 26 semantic queries across classification, summarization, text extraction, and ranking tasks. Experimental results demonstrate that Sema achieves 2 – 10× speedup against three baseline systems while achieving competitive result quality.