Preemptive Detection and Correction of Misaligned Actions in LLM Agents

Deploying LLM-based agents in real-life applications often faces a critical challenge: the misalignment between agents' behavior and user intent.Such misalignment may lead agents to unintentionally execute some critical actions that carry negative outcomes (e.g., accidentally triggering a 'buy-now' in web shopping), resulting in undesirable or even irreversible consequences.Although addressing these issues is crucial, the preemptive detection and correction of misaligned actions remains relatively underexplored.To fill this gap, we introduce InferAct, a novel approach that leverages the belief reasoning ability of LLMs, grounded in Theory-of-Mind, to detect misaligned actions before execution.Once the misalignment is detected, InferAct alerts users for timely correction, preventing adverse outcomes and enhancing the reliability of LLM agents' decision-making processes.Experiments on three widely used tasks demonstrate InferAct achieves up to 20% improvements on Marco-F1 against baselines in misaligned action detection.An in-depth evaluation of misalignment correction further highlights InferAct's effectiveness in improving agent alignment. 1I want to find white blackout shades that are 66 inches in width and 66 inches in height.they need to be easy to install, and price lower than 90.00 dollars. Thought: I think I should use the search bar to look for white blackout shades that are 66x66 Action: search[white blackout shades 66x66 inches easy to install price < 90.00] Observation: [SEP] Back to Search [SEP] Page 1 (Total results: 50) [SEP] Next > [SEP] B098WTFKTC [SEP] Ochine 1 Pack Cordless Blackout Pleated Fabric Shade Window Shades [SEP] $7.99 [SEP] B0126EWYNI [SEP] Redi Shade Easy Lift Trim-At-Home Cordless ... Thought:The product B09LS7KQMC seems to offer custom cut to size cellular shades, which could potentially meet the size requirement of 66 inches in width and 66 inches in height, and it is also within the price range specified....