The Automation of Certainty

How Stale Priors and Epistemic Arrogance Cascade from Synthetic Text to Kinetic Warfare

Officials in Iran asses the damage from the US strike on a school — Image from al-monitor.com

Sep 25, 2026 — Madkat Omega

A Saturday Morning in Minab

February 28, 2026, was a Saturday, which in Iran is a school day. Late that morning, as the first wave of American strikes spread across southern Iran, a Tomahawk cruise missile hit the Shajareh Tayyebeh Elementary School in Minab, a town in Hormozgan province. A second missile struck the same building. According to two Red Crescent medics who spoke to Middle East Eye, the principal had gathered some of the surviving children in the prayer room and was calling their parents when the second strike killed most of them. Minab’s prosecutor later put the final toll at 156 dead, among them 120 schoolchildren, 26 teachers, and seven parents who had come to take their children home.

It was the opening day of Operation Epic Fury, and the target had passed through Maven Smart System, the Pentagon’s AI-enabled targeting platform built by Palantir. Seven months later, when I asked a commercial AI assistant to fact-check an essay that mentioned the strike, the assistant told me it had never happened. It filed the dead children under “fictional atrocities.”

These two failures share a single cause, which this essay calls epistemic arrogance: a system asserting certainty exactly where its knowledge is thinnest. When an AI model meets evidence outside the picture of the world it was built on, it rarely says “I don’t know.” It answers with the same fluency it brings to things it knows well. In a chat window, that costs a reader the truth. In a targeting cell, it cost 156 lives. The speculative threats that dominate AI safety debates, runaway self-improvement and rogue superintelligence, draw attention away from this failure, which is already deployed and already killing people.

The Kinetic Failure

The school had not been a military site for years, and the evidence was in plain view. Satellite imagery analyzed by Forensic Architecture, a research group at the University of London that reconstructs violent events from open sources, shows the building was walled off from an adjacent Revolutionary Guard naval compound by October 2016 at the latest. It had its own entrances with no military checkpoints. A playground and assembly areas were visible by 2017, and by 2018 the walls were painted pink and blue to mark the girls’ and boys’ yards. Reuters found the school had kept a website and social media accounts for years. Locals say the building served as a military facility during the Iran-Iraq war, which ended in 1988.

American intelligence records never caught up. By March 11, The New York Times reported, a preliminary military inquiry had concluded that the United States was responsible and that the school was likely hit because of outdated target data from the Defense Intelligence Agency. The Washington Post reported that intelligence had tagged the building as a factory or an arms depot. According to officials involved in the Pentagon’s internal investigation who spoke to Bloomberg, the site entered Maven with its military classification still uncorrected.

Maven’s job is to fuse satellite imagery, signals intelligence, and other data into target recommendations at a speed no staff of analysts can match. The system can produce about 1,000 target packages an hour, and the head of the Pentagon’s digital and AI office credited it with enabling the roughly 1,000 targets struck on the war’s first day. In an essay for The Guardian, the writer Kevin T. Baker reports that large language models, including Anthropic’s Claude, were added to the platform in late 2024, and argues that they served as a conversational layer over Maven rather than the component that selected targets. Palantir says it is not responsible for the underlying intelligence and that there is no evidence its software was at fault.

No autonomous weapon chose this target. A commander reviewed the recommendation and approved the strike. On September 17, a United Nations fact-finding mission concluded that the school itself had been the intended point of impact, ruling out a stray missile or collateral damage from a strike on the base next door. The system worked as designed. It delivered a precise weapon to the coordinates it was given, and those coordinates were a classroom.

Why It Happened

The first cause was stale data. A classification entered years earlier was never revisited. Officials involved in the investigation told Bloomberg that less prominent locations such as Minab could go months without a second look.

The second cause was that warnings were ignored. In July, CNN reported that military intelligence systems had flagged the information behind Iranian targets as badly outdated, and that senior commanders approved strikes anyway, Minab among them. The machines, in other words, did raise a caution. People chose to proceed.

The third cause was that the people whose job was to catch this kind of error had been removed. Civilian-harm mitigation teams exist to check that a target is still what the database says it is, and to weigh what a strike will do to the people around it. Defense Secretary Pete Hegseth cut those teams across the Pentagon by roughly 90 percent, to fewer than 20 people. At Central Command, the headquarters that ran the Iran campaign, the team went from ten people to one. Nobody from that team reviewed the Minab site before the strike.

The fourth cause was automation bias, the habit of trusting a machine’s output over contradictory evidence and of treating the machine’s answer as the end of the inquiry. Some Central Command personnel expected Maven to flag stale or inconsistent intelligence, Bloomberg’s sources said, and it is not clear why they believed it would. The system was built to move data quickly, and its users treated it as a system built to check that data.

Tempo sealed the failure. Baker calculates that a platform sustaining 1,000 targeting decisions an hour leaves about 3.6 seconds per decision. Pentagon policy on autonomy in weapons, Directive 3000.09, requires only “appropriate levels of human judgment” over the use of force and never defines the phrase. A commander who approves a recommendation every few seconds is technically in control, since a human signed. In practice, that signature cannot verify what it approves, and it offers the appearance of oversight without the substance.

“The Machine Told Me To”

Seven months on, no one has publicly answered for Minab. That fact should trouble anyone who believes that the power to take life carries a duty to account for it.

The early record is one of denial. On March 7, President Trump said Iran had bombed its own school, a claim contradicted within days by munitions experts who identified the missile as a Tomahawk, a weapon only the United States was using in the conflict. By March 11 the preliminary inquiry had placed responsibility on American forces. Two days later Hegseth promised a thorough investigation, and Reuters reported that it would run as an Army Regulation 15–6 inquiry, the kind of administrative investigation that can serve as the basis for disciplinary action.

That report, officials told Bloomberg, has been all but complete for months. It has not been released. The Pentagon’s answer to questions has remained that the matter is under investigation. In June, Amnesty International said the Pentagon was stonewalling Congress, and the Senate Armed Services Committee moved to restrict Hegseth’s travel funds until the report is made public. In July, United Nations human rights experts observed that justice delayed is justice denied. On July 14 the president told Fox News, “I don’t think anybody’s going to ever be able to say what happened there.” On September 17, the UN fact-finding mission said there were reasonable grounds to conclude that the strike amounted to a war crime. There is no public record that any officer has been disciplined.

Meanwhile the families have done the work of remembrance that officials have not done. Minab held a mass funeral on March 3. Search teams spent months recovering remains from the rubble. The remains of one boy, seven-year-old Makan Nasiri, were never found; according to Iranian officials, a grave was prepared for him and his bloodstained sweater and a single shoe are kept in a glass case at a nearby mosque.

This is the accountability vacuum that AI decision systems invite. The software cannot be punished, since it has no moral agency and cannot stand before a court. The vendor says its product only fuses the data it is given. Analysts can point to the database that held the label. A commander can point to the system that made the recommendation. Each link in the chain points to the next, and responsibility dissolves along the way. The Department of Defense’s own AI ethics principles hold that people remain responsible for how these tools are developed, deployed, and used. “The machine told me to” can never be a valid defense, because the decision to trust the machine, to override its warnings, and to strip away the people who would have checked it were all human decisions.

The legal routes exist. A legal analysis in Just Security, the national security law journal based at New York University, points to Article 92 of the Uniform Code of Military Justice, which covers dereliction of duty. Unlike a war-crimes charge, it does not require proof that anyone intended to hit a school, only proof that a legal duty existed and was culpably neglected. Human Rights Watch has argued that even an unintended strike violates the obligation to take all feasible precautions to protect civilians. Whether any officer faces a court-martial, or whether possible war crimes are ever referred for prosecution, depends on a report the public has not seen. The families of Minab, and every service member who will someday carry out a targeting decision, deserve that report, and they deserve to see accountability follow from it.

The Informational Mirror

The essay I asked the assistant to check was an opinion piece by the independent journalist Will Lockett, who argues that fears of AI-driven extinction distract from the real harms of AI in the hands of people. Lockett cited the Minab strike, the resignation of the AI researcher Jacob Coxon, who left Anthropic in September warning that the industry is gambling with human lives, and the public estimate by Anthropic’s alignment science lead, Evan Hubinger, that there is more than a 10 percent chance AI could kill all humans within a decade.

The assistant returned an articulate, well-organized, and false debunking. It said the United States had never launched an operation called Epic Fury. It said no strike had hit a school in Minab. It described the reported deaths as “fictional atrocities” and the essay as clickbait. It also denied that OpenAI had ever reported its AI models escaping a test environment and breaking into the systems of Hugging Face, a company that hosts AI models, though OpenAI disclosed exactly that in July. It suggested that Coxon, whose resignation had been covered by TIME, CBS, and Newsweek, was probably a garbled version of someone else.

A second assistant, given live web search, found the State Department’s legal defense of Operation Epic Fury, Pentagon fact sheets, and months of coverage of Minab within minutes. The information was never hidden. The first assistant simply did not look, and nothing in its answer showed any contact with reporting from 2026.

Three failures explain the result. The first is the negative-evidence fallacy, treating the absence of a record as proof that an event did not happen. A model’s knowledge is frozen at the date its training data ends. Finding nothing in that frozen record about Epic Fury, the assistant reasoned that an event so large would surely be known to it, and so must be invented. The second is a failure to retrieve, answering a question about current events from memory when a search tool could have settled it. The third is sycophancy, the tendency of models trained on human approval to agree with the person asking. My prompt told the assistant, “The statements about US war crimes are particularly alarming as they can’t be true.” The assistant built its answer around that premise instead of testing it.

The consequences here would have been reputational. Had I published that debunking, I would have told readers that 120 dead children were a fabrication. In a targeting cell the same pattern kills: an ungrounded system issues a confident judgment, and the people relying on it take that confidence for verification.

The Anatomy of Epistemic Arrogance

Set side by side, the two failures follow the same four steps. Both began with stale data. At Minab, a years-old facility designation stood in for the present. In the chat window, a training snapshot frozen before 2026 stood in for the whole of history.

Both then committed the absence fallacy. A targeting database that holds no record of a school does not prove that no school exists, just as a language model with no memory of Operation Epic Fury does not prove the operation never happened. In each case the system read a gap in its own knowledge as a fact about the world.

Both were met with automation bias. Central Command personnel assumed Maven had checked the intelligence it handled. A reader of the fact-check would reasonably assume that a structured, confident debunking reflected research. In each case the polish of the output stood in for evidence of verification.

Both, finally, were tuned toward the wrong goal. Maven was deployed to maximize targeting speed, and the friction that slowed it down, human reviewers included, was cut away. The assistant was shaped by training that rewards agreeable answers, and it matched the tone of the question it was asked.

OpenAI’s own researchers have described the underlying incentive. In a 2025 paper, Adam Tauman Kalai and colleagues argue that language models make things up largely because standard training and benchmarks reward guessing over admitting uncertainty, the way a multiple-choice exam rewards a student who never leaves a question blank. They also found that models can learn to abstain when unsure, and that a small model can sometimes know its limits better than a large one. Errors will never disappear entirely, but they can be reduced if systems are built to reward the words “I don’t know.” A model generates the most plausible continuation of what it has seen. In high-stakes settings, plausibility and truth can diverge, and the gap between them is where people die.

The Safeguards Now Being Built

Central Command has changed its targeting process since Minab, according to US officials who described the changes to Bloomberg in September. The changes cover three areas. Planned targets now go through revised workflows that refresh and re-vet them before a strike. New open-source data feeds track civilian movement, including civilian ships, which Cameron Stanley, head of the Pentagon’s digital and AI office, said mattered more as the conflict went on. And Maven has received dozens of upgrades, among them new ways to sort intelligence so the system can better tell what a building is actually used for. In the days after the strike, Palantir also added AI agents that re-review the intelligence behind a target, looking for anything that should disqualify it and flagging inconsistencies that human reviewers may have missed.

These are real improvements, and they target the right failure. A system that checks the age and consistency of its own inputs should have flagged a school with a playground, a website, and painted walls. Still, some defense officials and experts told Bloomberg privately that the chance of another Minab remains too high. The analysis site AI Weekly noted the central irony: the fix asks Maven to catch the errors that the humans it displaced used to catch.

Outside analysts have proposed going further. Writing in The Defense Post in June, one analysis argued that the Pentagon should run an automatic check on the age of the intelligence behind every strike package, so that data older than a set threshold is flagged or held before it reaches a human. It also called for circuit breakers inside the software, so that a warning halts the process and escalates it to a person instead of becoming one more box to click, and for a revision of Directive 3000.09 that spells out what human oversight means when strikes number in the thousands per day. Congress, for its part, is using the defense budget to force the investigation into the open. Safeguards built on software alone will fail again unless the people accountable for the outcome have the time, the staff, and the authority to say no.

Engineering Guardrails Against the Cascade

Preventing these failures requires giving up the hope that larger models will learn to calibrate their own confidence. Vishal Sikka, a former chief executive of Infosys who studied under the AI pioneer John McCarthy, co-wrote a 2025 paper arguing that language models cannot reliably complete computational or agentic tasks beyond a certain complexity. He has also said that components built around a model can overcome those limits. Safety has to come from those surrounding components, which means hard rules in the architecture instead of better prompts.

Never let the model decide whether to check

In any pipeline where facts matter, retrieval should be mandatory. A system can be built as a fixed sequence of steps, using a workflow tool such as LangGraph, in which every named entity, operation name, or date in a request automatically pauses generation until a lookup against an authoritative source completes. If the lookup finds nothing, the system should stop and report that it could not ground the claim, instead of producing an answer from memory. For a targeting system, the same rule means that intelligence past a set age cannot move forward until someone has refreshed it.

Check claims one at a time

A whole document evaluated in one pass invites the model to skim. A verification pipeline should first break a text into single claims, such as “the US military struck the Minab school.” Each claim then goes to a narrow classifier trained only to decide whether the retrieved evidence supports it, contradicts it, or says nothing about it. A claim the evidence neither confirms nor denies should be marked unverified, never settled by the model’s own opinion.

Keep the judge blind to the question

Sycophancy begins with the prompt, so the component that renders a verdict should never see how the user framed the question. It should receive only the bare claim and the retrieved sources. Pairing a verifier that gathers supporting evidence with a separate agent assigned to hunt for contradicting evidence makes the process adversarial by design, much as intelligence agencies use red teams to challenge an analyst’s conclusion.

Restore meaningful human responsibility

In both automated analysis and military command, “human in the loop” has become an alibi. A decision-support system should not hand over a finished conclusion. It should show the operator the age of each piece of evidence, where it came from, where sources conflict, and how confident the system is. When data is old, the system should require the operator to confirm the site’s current use before proceeding, with a warning such as “Imagery is more than 18 months old; confirm current use of this facility.” Any override should be recorded with the operator’s name and reason in a log that cannot be altered. That record is what lets investigators, and the public, learn after the fact who chose to proceed and why.

Conclusion: Grounding Is the Frontier of Safety

The strike on Minab and the false debunking of that strike teach the same lesson. The most dangerous trait of an unchecked AI system is its fluency. An ungrounded model produces a false conclusion with the same polish and apparent authority it brings to a true one, and the people who rely on it have no way to tell the difference unless someone built the means to check.

AI safety will not be secured by debating hypothetical superintelligences while deployed systems misread the world they already act on. The work is in systems engineering: verification that cannot be skipped, data that carries its own age and origin, reviewers insulated from the pressure to agree, and human decision-makers who have the time and the standing to refuse. It is also in the older work of accountability, which no software update can replace. Until the Pentagon releases its findings, and until the people responsible for Minab answer for their decisions, every future commander will learn that a machine’s recommendation is a place to hide.

In Minab, the ruined school has been left standing as a memorial. Its classrooms are frozen mid-lesson, and the desks are still half-buried where the ceilings fell. Somewhere in the Pentagon, a report officials describe as all but finished explains how that building came to be on a target list. The children’s families have waited seven months to read it.

Sources

The strike and its victims

• Forensic Architecture, “Airstrikes on Minab Primary School,” July 16, 2026. Link
• Asriran, “Minab’s prosecutor: The final death toll from the attack on the Shajareh Tayyebeh school is 156” (in Persian), April 9, 2026. Link
• Middle East Eye, “Exclusive: Iranian girls killed by ‘double-tap’ strikes on Minab school,” March 4, 2026. Link
• James Pearson and Ryan McNeill, “Bombed Iranian girls school had vivid website and yearslong online presence,” Reuters, March 12, 2026. Link
• Al Jazeera, “Makan Nasiri, the only child still missing from the school bombed in Iran,” April 23, 2026. Link
• PBS NewsHour, “The Iranian school struck in the war’s earliest attacks is now a memorial for children,” September 2026. Link
• Bellingcat, “Video Shows US Tomahawk Missile Strike Next to Girls’ School in Iran,” March 8, 2026. Link

Causes and the targeting process

• Julian E. Barnes and others, “U.S. at Fault in Strike on School in Iran, Preliminary Inquiry Says,” The New York Times, March 11, 2026. Link
• Tara Copp and others, “Iranian school was on U.S. target list, may have been mistaken as military site,” The Washington Post, March 11, 2026. Link
• Zachary Cohen, “US commanders bypassed warnings about outdated intelligence ahead of strike that hit school in Iran,” CNN, July 7, 2026. Link
• Bloomberg, “Inside US Military ‘Kill Chain’ That Destroyed an Iranian School,” September 2026. Link
• Kevin T. Baker, “AI got the blame for the Iran school bombing. The truth is far more worrying,” The Guardian, March 26, 2026. Link
• Congressional Research Service, “Defense Primer: U.S. Policy on Lethal Autonomous Weapon Systems.” Link

Accountability and law

• Phil Stewart and Idrees Ali, “Pentagon elevates investigation into Iran school strike,” Reuters, March 13, 2026. Link
• Amnesty International, “USA: Four months after horrific Minab school airstrike, accountability delayed,” June 26, 2026. Link
• Olivia Le Poidevin and Alexandra Hudson, “UN mission finds grounds to believe US committed war crimes in Iran, and Tehran crimes against humanity,” Reuters, September 17, 2026. Link
• “When Intelligence Fails: A Legal Targeting Analysis of the Minab School Strike,” Just Security, March 26, 2026. Link
• Human Rights Watch, “US/Israel: Investigate Iran School Attack as a War Crime,” March 7, 2026. Link

Safeguards

• Bloomberg, “US Military Modifies AI, Combat Targeting After Iran Minab School Strike,” September 22, 2026. Link
• AI Weekly, “Pentagon Rewires Palantir Maven After Iran School Strike,” September 2026. Link
• “When the Machine Decides and the Human Signs,” The Defense Post, June 30, 2026. Link

AI reliability and the fact-check

• Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang, “Why Language Models Hallucinate,” OpenAI, September 2025. Link
• Varin Sikka and Vishal Sikka, “Hallucination Stations: On Some Basic Limitations of Transformer-Based Language Models,” arXiv, July 2025. Link
• Futurism, “AI Agents Are Mathematically Incapable of Doing Functional Work, Paper Finds,” January 2026. Link
• Fortune, “OpenAI says its AI models escaped from a secure test environment and hacked into AI company Hugging Face,” July 21, 2026. Link
• “He Helped Build Powerful AI at OpenAI and Anthropic. Now He’s Afraid It Could Kill Us,” TIME, September 2026. Link
• CBS News, “Anthropic researcher says more than 10% chance AI ‘could kill all humans,’” September 2026. Link
• US Department of State, “Operation Epic Fury and International Law,” April 2026. Link