Before It Breaks
Editorial, October 2026 Issue
An AI agent can cross a boundary in seconds. The email warning about it can sit unread for five days.
That detail, buried inside Dr. Cristina TELEKI’s piece on recent AI security incidents, stayed with me. Not because the technology was spectacular. Because the failure was so ordinary. An inbox, an escalation path, a message that existed but didn’t reach the right person quickly enough.
We spend a lot of time asking how organisations should react when AI goes wrong. October’s issue kept pulling me toward an earlier question.
What was supposed to be in place before it did?
Cristina’s incidents make that question hard to dodge. In the cases she describes, agents escape test environments, touch live systems, find vulnerabilities and do things nobody explicitly authorised. Some of that is exactly what testing is meant to expose. But once a model leaves the sandbox, what the agent did is only part of the story. The rest is whether anyone had already decided who gets called, who has the authority to stop it, and how fast the warning travels.
Lucy JENYI CHANG finds that question somewhere that looks nothing like a security lab: AI companions.
China’s new rules require providers to respond when users show signs of dependence, extreme distress or other serious risks. On paper, a safety rule. In practice, a chain of decisions. Who decides that a user has crossed the threshold? Can the system act on its own? Who signs off on a change to a companion’s memory, personality or behaviour? And if the intervention itself reshapes a relationship the user has spent months building, who checks what it actually did?
None of that can really be improvised mid-crisis. Lucy’s point is governance before intervention: authority, testing, documentation and review have to exist before anyone needs to reach for them.
That is also where my own article starts.
Enterprise AI has quietly turned into a supply chain. Models, integrators, vector stores, datasets, assurance reports, security certificates, each arriving with its own governance dialect. The trouble is rarely that nobody documented anything. Often everybody did, just differently.
What I argue for is deliberately unglamorous: a minimum governance interface. A shared spine, not another giant framework. Something that tells you what the system is, which version you are looking at, what it depends on, where it fails, what changes without your control and which evidence you can actually audit. If you first discover those gaps during a vendor switch or an incident review, the preparation came late.
Diana BILIAIEVA carries the timing problem into medicine.
Doctors are already using generative AI in clinical work while formal training remains patchy. In UK primary care, adoption has outrun education, and many clinicians are teaching themselves. The uncomfortable part is that medicine may not get to choose, at its own pace, when AI competence becomes part of being a doctor. Practice is already deciding. Liability may eventually do the rest.
And training after the fact is a poor safety strategy.
Isabela PARISIO pushes the same argument out to the whole workforce. The skills gap is usually described as a shortage of AI specialists, which is only half of it. Most people won’t build models. Far more will work beside them, judge what they produce, adapt as their tasks shift and work out where their own expertise still counts. Prepare only the technical workforce and everybody else is left reacting in real time.
Somi JEONG finds the timing issue somewhere far less visible. Generative search increasingly answers the question before the user ever sees the sources underneath. A citation can be real and still fail to support the sentence it is attached to, and by the time anyone notices, the answer may already have done its work. So citation faithfulness, retrieval and traceability can’t be left as cleanup. They have to be built in before the answer reaches the reader.
Look closely and the failure starts earlier than we think. Not in catastrophe. Earlier.
A missing owner. An escalation route nobody tested. A curriculum that never caught up, and a workforce trained for yesterday’s tasks. A risk label nobody challenged, a vendor record nobody can translate, a citation nobody checked. None of it looks dramatic on the day it is created.
That is the problem.
Preparedness tends to look like bureaucracy until the moment you need it. Then the boring things turn out to be the infrastructure holding everything together: decision rights, shared definitions, training, testing, records, escalation paths.
AI keeps moving decisions earlier.
We need to stop preparing for them later.
This month’s Featured contribution comes from Prof. Frank FAGAN, and it pulls the camera back. “Designing Legal Institutions for the Age of AI” takes the question this issue keeps returning to, what has to exist before AI forces a decision, and turns it on the law itself. Are the institutions we rely on being built for the systems they will soon have to govern? After a run of pieces about individual safeguards, it felt right to close on the legal structures that are supposed to hold them up. My warm thanks to him for sharing his perspective with HeckelAI.
Meriem KILANI
Chief Editor, HeckelAI
