AFE-Master: Enhancing LLM-Driven Autonomous Feature Engineering with Domain-Specific Language Parsing and Guided Local Search

Autonomous Feature Engineering (AFE) is critical for improving predictive performance on tabular data by relieving humans from manual feature crafting. However, traditional AFE lacks the semantic guidance needed to fully exploit domain knowledge. Although large language models (LLMs) can, in principle, emulate experts, existing approaches typically operate in an open code space that directly generates and rewrites entire features; without a compositional structural representation and invariant constraints, edits are coarse and non-local, making it hard to distill interpretable features with high information content and rich hierarchical structure. We propose AFE-Master, a novel LLM-driven AFE framework that shifts feature construction from black-box evolution to structural and semantically rigorous search. AFE-Master uses a domain-specific language (DSL) to explicitly represent feature transformations and parses them into abstract syntax trees (ASTs), enabling the LLM to understand and manipulate feature structures with clear semantics. On this interpretable representation, we employ guided local search (GLS) over syntactic and semantic neighborhoods, making small, checkable edits that efficiently and controllably discover information-dense, hierarchically structured features. Experiments spanning Kaggle and OpenML benchmarks as well as multiple tabular models (XGBoost, MLP, and the frontier TabPFN) show significant gains. At industrial scale, we further conduct a large online A/B test on the advertising recommendation service of a major mobile app store. Starting from a mature, large feature set—167 expert-crafted features refined over two years—we add 20 features automatically generated by AFE-Master. On 100M+ live samples, a well-engineered FiBiNET model achieves +15.11% CPM and +3.01% CTR, demonstrating practical value and transferability under both massive sample volume and a high-feature-count production setting. These results indicate that AFE-Master's semantically guided approach can discover expert-level features beyond the reach of prior methods, pointing to a new generation of interpretable, high-performance AFE techniques.