The convening body for AI safety data

A Pragmatic Classification Framework for AI Incident Monitoring addresses a core problem in AI governance: raw incident counts conflate media reporting propensity, system deployment, and actual harm frequency. The framework offers a practical, evidence-calibrated way to read incident data despite those confounders.

Three components

• A structured monitoring question that defines the analytical scope.
• A tiered estimation process that uses LLM-assisted filtering to analyze harm and exposure trends separately.
• A classification scheme that labels findings as Escalating, Mitigating, Concentrating, Receding, or Unclassifiable.

The authors demonstrate the approach through real-world case studies, working within the data limitations typical of incident monitoring. Read the full paper on arXiv.

AI Incident Monitoring through a Public Health Lens adapts public-health surveillance methods — which presume noisy, incomplete data — to the problem of measuring AI risk. Incident databases index what has happened, but they cannot express risk (a joint measure of likelihood and severity) without knowing how prevalent risk-associated systems are and how often incidents are reported.

The framework

The paper identifies six phases of incident emergence and shows how an informed panel of domain experts can combine that framework with incident data and statistical and visualization tools to arrive at phase determinations that serve public needs. It is demonstrated through:

• Autonomous vehicles, whose mandatory reporting requirements produce reliable incident-rate ground truth (expressed in distance traveled).
• Deepfakes, as a second case study, before charting a path for future incident-phase research.

For a short overview, see the abstract and executive summary — Phase Modeling for AI Incident Emergence: Adapting Epidemiological Methods to Post-Deployment Governance. The full paper is on arXiv.

This Springer reference-work chapter offers a broad, citable overview of how AI risk and safety connect to the practice of incident reporting — why real-world harm events are worth cataloging, how incident databases are built and maintained, and how reporting fits into the wider landscape of AI governance.

It is a useful entry point for readers approaching AI incident reporting for the first time.

Read it via Springer.

Curating an incident database is as much an editorial practice as a technical one. This AAAI 2025 paper distills lessons from the people who maintain the AI Incident Database — how to decide what counts as an incident, how to handle ambiguous or contested reports, and how to keep classifications consistent as the collection and its editors grow.

It is a practical companion for anyone building or operating a real-world AI harm catalog.

Read it at AAAI.

After two years operating the AI Incident Database, a backlog of issues — cases that reveal AI risks but do not meet the bar for an incident — accumulated. Drawing on lessons from editing 2,000+ incident reports, this paper proposes a two-tiered system, as used in aviation and computer security, that distinguishes:

• Incidents — realized harms or near-harms in the real world.
• Issues — credible risks where harm has not (yet) occurred.

It also introduces incident variants for machine learning systems that produce many similar incidents, so repeated failure patterns can be grouped rather than re-litigated.

Read it on arXiv.

This AAAI/IAAI 2021 paper introduces the AI Incident Database (AIID) — a collection of more than a thousand reports of intelligent systems causing safety, fairness, or other real-world problems. Modeled on the incident databases of aviation and computer security, it argues that systematically cataloging failures is a precondition for not repeating them.

The paper describes the database's motivation, its data model of incidents and reports, and the application architecture that makes the collection searchable and extensible.

Read it at AAAI.

Across three rounds, 272 international experts ranked which AI threats most warrant urgent attention, weighing likelihood, magnitude of harm, who is vulnerable, and who bears responsibility.

Findings

• The five most concerning near-term harms were dangerous capabilities, competitive dynamics, weapons and cyberattacks, power concentration, and misinformation.
• Under business-as-usual conditions, eighteen of twenty-four assessed risks carried more than a 10% chance of catastrophic outcomes (over one million deaths or $100 billion in losses) between 2025 and 2030.
• Experts placed primary responsibility for mitigation on general-purpose AI developers and governance institutions.

Read it on arXiv.

Reporting practices borrowed from traditional cybersecurity and general AI safety are not well aligned with the distinctive characteristics of AI security incidents. This SaTML 2026 position paper maps where those frameworks fall short — including unresolved questions of intellectual property and vulnerability ownership — and separates issues that are immediately addressable from those that remain open.

The authors argue that purpose-built AI incident reporting standards are urgently needed, and that the rise of autonomous AI agents will only sharpen that need.

The full paper is on arXiv; the published version appears in IEEE SaTML 2026.

Evaluation conducted by AI developers alone cannot surface the flaws that matter once systems are deployed at scale. This ICML 2025 position paper — with contributors from more than two dozen institutions — proposes three interventions:

1. Standardized flaw reports and researcher-engagement guidelines.
2. Industry-wide disclosure programs, modeled on cybersecurity bug bounties, with legal safe harbors to protect researchers.
3. Coordination infrastructure to route vulnerability information to all affected parties.

Transferable flaws — jailbreaks that work across multiple providers — show why disclosure mechanisms comparable to established software-security practice are overdue.

Read it on OpenReview.