Regression Aware Inference with LLMs

Large language models (LLMs) have shown strong results on a range of applications, including regression and scoring tasks.Typically, one obtains outputs from an LLM via autoregressive sampling from the model's output distribution.We show that this inference strategy can be sub-optimal for common regression and scoring evaluation metrics.As a remedy, we build on prior work on Minimum Bayes Risk decoding, and propose alternate inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses.We show that our proposal significantly improves over baselines across datasets and models.p(y|x) LLM y 1 y 2 y 3 y 4 y 5 sampling from p(y|x) forward passWhat is the rating corresponding to the following review on the scale of 1 to 5? Review: this product is great!Rating: Decision rule