Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles

Autonomous vehicles (AVs) need to communicate with other road users in uncertain scenarios via external human-machine interfaces (eHMIs). Existing eHMIs are rule-based, relying on manually designed motion and communication patterns for predefined scenarios, which limits adaptability. To address this, we explore LLM-driven eHMIs, a new approach that leverages the human-like reasoning and communication abilities of Large Language Models (LLMs) for generating expressive communicative actions. We investigated two research questions: 1) How does a pre-trained LLM translate intended communicative messages into corresponding eHMI actions? and 2) How communication-efficient are these generated actions to humans? To answer these questions, we first constructed a prompt framework through iterative prototyping, followed by the development of an end-to-end physical pipeline. Finally, we conducted a 26-participant field study using a physical prototype to assess communication effectiveness by comparing human-designed and GPT-o3-mini-generated eHMI actions in terms of participants' correct reactions and message interpretation. Results show that participants' correct reactions do not equate to their correct interpretations. The current pre-trained LLMs can produce communicative actions with similar interpretation comparable to human-designed ones, but cannot enable the correct pedestrian reactions. Our discussion highlights different design patterns between humans and LLMs.