Density estimation with LLMs: a geometric investigation of in-context learning trajectories

Large language models (LLMs) demonstrate remarkable emergent abilities to perform in-context learning across various tasks, including time-series forecasting. This work investigates LLMs' ability to estimate probability density functions (PDFs) from data observed in-context; such density estimation (DE) is a fundamental task underlying many probabilistic modeling problems. We use Intensive Principal Component Analysis (InPCA) to visualize and analyze the in-context learning dynamics of LLaMA-2, Gemma, and Mistral. Our main finding is that these LLMs all follow similar learning trajectories in a low-dimensional InPCA space, which are distinct from those of traditional density estimation methods such as histograms and Gaussian kernel density estimation (KDE). We interpret these LLMs' in-context DE process as a KDE algorithm with adaptive kernel width and shape. This custom kernel model captures a significant portion of the LLMs' behavior despite having only two parameters. We further speculate on why the LLMs' kernel width and shape differ from classical algorithms, providing insights into the mechanism of in-context probabilistic reasoning in LLMs. Our codebase, along with a 3D visualization of an LLM's in-context learning trajectory, is publicly available at https://github.com/AntonioLiu97/LLMICL_inPCA.