OntoGuard: Enforcing Action Admissibility for LLM Agents in Complex Interactive Environments

Large Language Models (LLMs) have shown impressive reasoning capabilities in agents for complex interactive environments. However, these agents often suffer from hallucinations and lack grounding, leading to unreliable actions that conflict with real-world constraints. Existing approaches mitigate some issues through implicit imitation or sparse reinforcement learning but rely on fitting data distributions without explicitly understanding environmental constraints, often generating actions that are behaviorally distorted or environmentally impermissible. To address this, we introduce OntoGuard, an ontological framework designed to guard LLM agents by enforcing environmental and behavioral admissibility. These constraints are constructed by extracting knowledge from oracle demonstrations, supplemented with world knowledge inherent in LLMs and general knowledge bases. During inference, OntoGuard functions as an active interceptor, using a graph-based constraintchecking mechanism to reject invalid actions and prompt self-correction before acting. Experiments on both ScienceWorld and Virtu-alHome demonstrate OntoGuard's advantage over state-of-the-art methods, validating its ability to enforce physical and behavioral constraints while preventing invalid actions.