NordVPN Promotion

Home / Blogs

Safe for Whom? A Pilot’s Take on the AI Incident-Reporting System

Aviation learned that trust encourages disclosure. AI’s emerging incident-reporting system will succeed only if confidential reports produce public lessons and broader legitimacy.

Earlier in my life, I flew myself around California, and once I had to file a report with NASA’s Aviation Safety Reporting System after briefly clipping the corner of a Class B airspace.

The cause was an ordinary but consequential communications failure. The part of an air traffic control transmission distinguishing “you are cleared” from “remain clear” was stepped on by another radio call, and I understood I had been cleared into the airspace when I had in fact been told to stay out. Nothing dramatic happened. It mattered anyway, because the same mix of ambiguous wording, overlapping transmissions and reasonable human interpretation could recur under less forgiving circumstances, and that is exactly what a safety-reporting system exists to capture: not disasters, but the near misses and small failures that reveal weaknesses before they produce consequences.

I reported it because ASRS offered a credible place to tell the truth. The report went not to the regulator or to an employer with a commercial stake in the outcome, but to NASA, a public institution with no enforcement power and no interest in assigning blame. My identity was protected, and filing promptly gave me limited immunity for an inadvertent violation. The FAA had, in effect, agreed that learning mattered more than punishment. That arrangement is the heart of the system: people disclose problems voluntarily only when they believe the knowledge will make things safer, including for themselves.

From Aviation Safety to AI Safety

I was reminded of all this while reading the Linux Foundation’s proposal for a Shared AI Findings Exchange, or SAFE, drafted by a working group of the Open Secure AI Alliance and framed mainly around AI security incidents. The analogy with ASRS is sound: AI developers and deployers need somewhere to share failures without every report becoming an enforcement action, a lawsuit or a public-relations crisis.

But the analogy also reveals what is missing. ASRS works because its trusted intermediary is public, non-regulatory and non-commercial: NASA, not the FAA, and not the airlines. In the SAFE proposal, the system is built around participating organizations, with governments and standards bodies admitted as “non-controlling observers,” and the closest analog to NASA is absent: no international public institution appears anywhere in the document. I spent the second part of my career in the United Nations system working on technology and development, so I notice the absence, and what those institutions could bring: universal membership, public legitimacy, and reach into the countries where AI deployment harms will concentrate.

Designing Legitimacy from the Start

This matters because SAFE, if it succeeds, will shape how AI incidents are defined, categorized, investigated and turned into lessons. Those are governance functions, whether or not anyone calls them governance, and the internet has been through this before. In 1998, the United States government published its Green Paper on the management of internet names and addresses; the process that followed produced ICANN and private-sector-led coordination of critical infrastructure. The mistake was not private-sector leadership; speed, technical competence and insulation from governmental machinery were real strengths. The mistake was assuming that international legitimacy, accountability and participation could be added later without great difficulty. It took until the 2016 IANA stewardship transition, roughly twenty years, for legitimacy to catch up with function. The lesson is not that industry should wait for governments to agree among themselves; industry builds faster, SAFE appeared a week after its alliance formed, and it should keep building. The lesson is that speed and legitimacy have to be designed together, not sequenced.

A reporting system created mainly by organizations close to the leading AI markets will naturally reflect their concerns, vocabulary and institutional habits. That is not bad intent; it is simply how systems develop. This is why geographic and institutional diversity should not wait until after SAFE’s pilot phase, and why access to the testing phase should be equitable across represented countries rather than sequenced by proximity to the founding members. By the end of a pilot, the categories, workflows and trust relationships will have taken shape, and the accumulated evidence, the system’s most valuable asset, will be compounding for those inside. A commitment to inclusive participation costs one sentence now, and much more once the system is operational.

Confidential Reports, Public Lessons

The second issue is disclosure. The proposal anticipates that some findings and defensive recommendations may eventually be published, subject to safety considerations, which raises the obvious question: safe for whom? The company whose system failed, the users still exposed, the researchers trying to understand the vulnerability? There are legitimate reasons to withhold information, but reputational damage and commercial inconvenience should not quietly become equivalent to public safety. ASRS answers this too: individual reports stay confidential, while the database and its CALLBACK bulletins are open to every pilot, not just to airline employees. AI development is not containable to a membership roster; it happens wherever weights and compute exist, and if the lessons stay inside a closed network, the open community keeps making mistakes the members already understand, and the blind spot eventually returns as the members’ own exposure. The default should be simple: reports confidential, lessons public, with exceptions narrow, documented, reviewable and time-limited. The organization involved should be able to correct errors and flag concrete risks; it should not be the sole judge of whether publication is safe.

A Role for Public-Interest Institutions

None of this requires turning SAFE into an intergovernmental body, and international participation should not mean veto power or access to confidential reports. It means something practical: standing for public-interest institutions to help shape incident taxonomies, machine-readable schemas, test design and the distribution of de-identified lessons. The candidates exist. The OECD already runs an AI Incidents Monitor and has published work toward a common reporting framework; the UN’s new Global Dialogue on AI Governance and its Independent International Scientific Panel need exactly the evidence base SAFE will accumulate; the ITU and regional bodies can carry the lessons well beyond the founding membership. And here I address my own side of this conversation: those institutions need to move past adoption language and summit rhetoric to a concrete act, formally request access to SAFE’s aggregated learning and offer schema convergence, distribution and capacity building in return. I intend to make that argument to UN interlocutors just as I make this one here. The appropriate institutional response to a request for comments is comments.

Whoever builds this system will eventually define what counts as an incident, what evidence matters and what remains unseen. Retrofitting legitimacy cost the internet twenty years; designing it in at the RFC stage costs a paragraph.

NORDVPN DISCOUNT - CircleID x NordVPN
Get NordVPN  [74% +3 extra months, from $2.99/month]
By Gabriel Accascina, Independent Senior Advisor

Gabriel Accascina is the former Director of the Knowledge Management Group at the United Nations Development Programme (UNDP) and a former Fellow at Harvard Kennedy School’s Center for International Development. Active in internet governance since its formative years, he contributed to the 1998 U.S. Department of Commerce Green Paper that laid the foundation for ICANN. Outside technology and public policy, he also holds a commercial pilot’s license.

Visit Page

Filed Under

Comments

Comment Title:

  Notify me of follow-up comments

We encourage you to post comments and engage in discussions that advance this post through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can report it using the link at the end of each comment. Views expressed in the comments do not represent those of CircleID. For more information on our comment policy, see Codes of Conduct.

CircleID Newsletter The Weekly Wrap

More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

Related

Topics

DNS

Sponsored byDNIB.com

DNS Security

Sponsored byWhoisXML API

Cybersecurity

Sponsored byVerisign

New TLDs

Sponsored byRadix

Domain Names

Sponsored byVerisign

IPv4 Markets

Sponsored byIPv4.Global

Brand Protection

Sponsored byCSC

NordVPN Promotion