ACL Findings2026
SafetyALFRED: Evaluating Safety-Conscious Planning of Vision Language Models
Josue Torres-Fonseca, Naihao Deng, Yinpei Dai, Shane Storks, Yichi Zhang, Rada Mihalcea, Casey Kennington, Joyce Chai
Abstract
Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insufficient. We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards. While existing safety evaluations focus on hazard recognition through disembodied question answering (QA) settings, we evaluate eleven state-of-the-art models from the Qwen, Gemma, and Gemini families on not only hazard recognition, but also active risk mitigation through embodied planning. Our experimental results reveal a significant alignment gap: while models can accurately recognize hazards in QA settings, average mitigation success rates for these hazards are low in comparison. Our findings demonstrate that static evaluations through QA are insufficient for physical safety, thus we advocate for a paradigm shift toward benchmarks that prioritize corrective actions in embodied contexts. We open-source our code and dataset at: https://github. com/sled-group/SafetyALFRED.git
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on10
- Safe RLHF: Safe Reinforcement Learning from Human FeedbackJosef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji et al.ICLR 2024 · 656 citations
- Episodic Transformer for Vision-and-Language NavigationAlexander Pashevich, Cordelia Schmid, Chen SunICCV 2021 · 228 citations
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language ModelsShengyun Peng, Pin-Yu Chen, Matthew Hull, Duen Horng ChauNeurIPS 2024 · 68 citations
- HAZARD Challenge: Embodied Decision Making in Dynamically Changing EnvironmentsQinhong Zhou, Sunli Chen, Yisong Wang, Haozhe Xu et al.ICLR 2024 · 33 citations
- IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household TasksXiaoya Lu, Zeren Chen, Xuhao Hu, Yijin Zhou et al.AAAI 2026 · 20 citations
Related papers
- Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision MakingYejin Son, Minseo Kim, Sungwoong Kim, Seungju Han et al.EMNLP 2025 · 7 citations
- AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous InstructionsZonghao Ying, Le Wang, Yisong Xiao, Jiakai Wang et al.CVPR 2026 · 42 citations
- LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied AgentsJae-Woo Choi, Youngwoo Yoon, Hyobin Ong, Jaehong Kim et al.ICLR 2024 · 49 citations
- EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied AgentsDongwook Choi, Taeyoon Kwon, Bogyung Jeong, Minju Kim et al.ICML 2026
- When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily LifeXinyue Lou, Jinan Xu, Jingyi Yin, Xiaolong Wang et al.ACL Findings 2026 · 1 citation
