TabLoft: Tabular Data Generation Based on LLM with Ordered Features

Tabular data is a fundamental format for storage and processing in databases. However, it often suffers from challenges such as limited sample size and privacy concerns in specific tasks. Tabular data generation offers a promising solution, and methods based on large language models (LLMs) have demonstrated strong performance. However, existing methods do not account for the unidirectional propagation inherent in the transformer architecture of LLMs. Consequently, the generation of preceding tokens cannot benefit from information in subsequent tokens, resulting the ordering of tabular features particularly critical. To address this limitation, we propose TabLoft, a Tabular data generation method based on LLM with ordered features. Inspired by the effectiveness of tree-based models in tabular tasks, we conduct a systematic analysis and design a feature ordering strategy accordingly. The ordered data is then encoded and used to fine-tune the LLM, enabling the generation of high-quality data. We introduce five evaluation metrics to quantitatively assess the quality of generated tabular data. The experimental results demonstrate that TabLoft achieves state-of-the-art performance compared to six baseline models.