FLARE-AI: Flaw Reporting for AI
Shayne Longpre, Elaine Zhu, Carson Ezell, Avijit Ghosh, Sean McGregor, Kevin Paeth, Kevin Klyman, Sayash Kapoor, Rishi Bommasani, Ruth Elisabeth Appel, Gregory Strom, Lauren McIlvenny
Abstract
Flaw reporting for deployed AI systems is fundamental to identifying system failures and improving AI safety. Yet the AI reporting ecosystem is fragmented: researchers who identify flaws often do not know what or where to report, and groups who receive reports rarely share them with other relevant stakeholders. As a result, good-faith reporters duplicate effort by submitting many different forms, and recipients lack standardized, triage-ready information. We survey 12 reporting systems published by AI developers, cybersecurity groups, and AI flaw aggregators, identifying five recurring design challenges spanning discoverability, scope, information collection, coordination, and guidance for strict-liability cases. Building on this analysis and feedback from 49 experts across 32 organizations representing developers, security researchers, and ecosystem coordinators, we introduce FLARE-AI, an open-source AI flaw reporting system designed for interoperability with existing systems. FLARE-AI streamlines flaw report creation by collecting triage-relevant information through conditional logic and early classification, then enables optional dissemination of standardized, machine-readable reports to multiple developers, coordinators, and incident registries from a single submission. By lowering barriers to reporting AI flaws and improving interoperability across stakeholders, FLARE-AI helps break down silos and accelerate remediation across the AI ecosystem.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 5bebe153-7600-4c9e-b845-60d744b46fc8Builds on7
- Adversarial NLI: A New Benchmark for Natural Language UnderstandingYixin Nie, Adina Williams, Emily Dinan, Mohit Bansal et al.ACL 2020 · 602 citations
- Stealing part of a production language modelNicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke et al.ICML 2024 · 157 citations
- Best-of-N JailbreakingJohn Hughes, Sara Price, Aengus Lynch, Rylan Schaeffer et al.NeurIPS 2025 · 78 citations
- Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and AgencyKyra Yee, Uthaipon Tantipongpipat, Shubhanshu MishraCSCW 2021 · 52 citations
- Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party EvaluationsAnka Reuel, Avijit Ghosh, Jenny Chim, Andrew Tran et al.ICML 2026 · 9 citations
Related papers
- Discovering and Validating AI Errors With Crowdsourced Failure ReportsÁngel Alexander Cabrera, Abraham J. Druck, Jason I. Hong, Adam PererCSCW 2021 · 60 citations
- Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit ToolingVictor Ojewale, Ryan Steed, Briana Vecchione, Abeba Birhane et al.CHI 2025 · 46 citations
- "Abuse Risks are Often Inherent to Product Features": Exploring AI Vendors' Bug Bounty and Responsible Disclosure PoliciesYangheran Piao, Jingjie Li, Daniel W. WoodsUSENIX Security 2026 · 1 citation
- Actively Understanding the Dynamics and Risks of the Threat Intelligence EcosystemTillson Galloway, Omar Alrawi, Allen Chang, Athanasios Avgetidis et al.NDSS 2026 · 2 citations
- Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?The Anh Nguyen, Triet Huynh Minh Le, M. Ali BabarICSE 2026 · 1 citation
