EnCompass: Enhancing Agent Programming with Search Over Program Execution Paths

We introduce a new approach to agent programming, the development of LLMbased agents. Current approaches to agent programming often entangle two aspects of agent design: the core workflow logic and the inference-time strategy (e.g., tree search). We introduce probabilistic angelic nondeterminism (PAN), a programming model that disentangles these two concerns, allowing the programmer to describe the agent workflow and independently experiment with different inference-time strategies by simply changing a few inputs. We provide an implementation of PAN in Python as the ENCOMPASS framework, which uses a Python decorator to compile agent workflow programs into a search space. We present three case studies that demonstrate how the framework lets the programmer quickly improve the reliability of an agent and easily switch between different inference-time strategies, all with little additional coding.

• Work performed as a consultant for Asari AI Core primitives The two most important primitives that are available in the body of an ENCOM-PASS-decorated function are branchpoint() and record_score(). branchpoint(**branchpoint_params)