Why “human-in-the-loop” is the most dangerous lie in modern engineering
Marcus, locked inside his own creation — Image by Gemini
Oct. 19, 2026 — Madkat Omega
Marcus did not start using autonomous coding agents because he was lazy. He started using them because he was ambitious.
At thirty-two, he was the kind of systems engineer every organization quietly depends on: the guy who could read a flame graph like sheet music, who understood how memory pages actually behaved under kernel pressure, and who felt a visceral, almost physical satisfaction when a stubborn race condition finally yielded at two in the morning. He loved the craft. He loved the metal.
When the first generation of AI assistants arrived, Marcus treated them like power tools. Why waste forty minutes writing boilerplate serialization logic or scaffolding test harnesses when an model could spit them out in four seconds? Freeing himself from the drudgery meant he could spend more time on high-level systems design. It was a rational trade. It was an obvious trade.
Within eighteen months, Marcus’s output had quadrupled. Within thirty months, he was promoted twice, overseeing a sprawling fleet of agentic workflows that drafted, tested, and deployed distributed services across the company’s infrastructure.
He was faster than he had ever been. He was more valuable on paper than he had ever been. And without realizing it, he had walked step by logical step through three distinct stages of algorithmic surrender — ending up trapped inside a machine he had personally helped build, but no longer knew how to escape.
We talk about the future of automation as if it were a binary choice between utopian abundance and mass unemployment. But for those of us who actually build and maintain the digital world, the real trajectory is much stranger. It is a three-act progression of alienation — one that science fiction has already mapped out in terrifying detail, and one that you are likely living through right now.
Stage I: The WALL-E Phase (The Atrophy of Friction)
The first stage never feels like a loss. It feels like liberation.
When people reference Pixar’s WALL-E, they usually point to the sight gag of future humans lounging in hovering recliners, sipping liquified meals through straws while staring at screens inches from their faces. It is easy to laugh at the image because it looks like gluttony.
But the tragedy of the Axiom wasn’t born out of hedonism; it was born out of relentless optimization. Over generations, the ship’s systems simply removed every point of physical and cognitive friction from human life. Walking requires effort; hovering is seamless. Searching requires patience; instant delivery is efficient. Nobody aboard that ship voted to let their bones calcify and their curiosity wither. They just accepted a series of small, frictionless conveniences until the capacity for independent action quietly evaporated.
In software and systems engineering, we have been sold the exact same doctrine: friction is a bug.
When Marcus began delegating his daily work to multi-agent coding pipelines, he started by handing off the tedious parts — the unit tests, the configuration manifests, the data migrations. Soon, however, the agents got good enough to handle the core logic. Instead of wrestling with an unfamiliar concurrency model or digging through source code to understand why a library behaved strangely under load, Marcus simply described the desired outcome in a prompt, went to pour a cup of coffee, and returned to a clean, green pull request.
On the company’s engineering velocity dashboard, every metric turned bright green. Cycle time plummeted. Deployment frequency soared.
What the dashboard could not measure was what vanished along with the friction.
In any complex craft, friction is not wasted heat. Friction is the epistemic mechanism by which a human builds a mental model of a system. When you write flawed code, watch it fail, trace the stack, curse at an obscure documentation page, and rewrite the function three times, you aren’t just producing syntax. You are mapping the invisible topography of the architecture into your own nervous system. You are learning where the brittle edges are, how state flows through the pipes, and what assumptions the system makes when things go wrong.
When Marcus eliminated the struggle of building, he eliminated the mechanism of understanding. His fingers forgot the texture of the codebase. His architectural intuition — once forged by thousands of hours of debugging — began to atrophy from disuse. He was still sitting at the helm, issuing high-level commands from his ergonomic chair, feeling remarkably productive. He didn’t notice that he had stopped walking.
Stage II: The Severance Phase (The Compartmentalization of Meaning)
Once an organization automates the act of creation, it inevitably faces a governance crisis: machines hallucinate, optimize for the wrong variables, and introduce subtle vulnerabilities. To solve this, corporate leadership invokes the most soothing mantra in modern technology:
“Don’t worry. We keep the human in the loop.”
This brings us to the second stage of the progression — the world of Severance.
On the surface, the horror of Lumon Industries is the surgical division of memory between work and home. But look closer at what the employees on the Macrodata Refinement floor actually do all day. They sit at terminals staring at seas of numbers, waiting to feel an intuitive, emotional reaction to a cluster of digits, and then drag those digits into digital bins. They are working hard. They have quotas, performance reviews, and strict compliance protocols. Yet they are completely severed from context. They do not know where the numbers come from, what the bins trigger, or what they are building.
This is the exact reality of the modern “human-in-the-loop.”
As Marcus’s organization scaled its agentic pipelines, his job title evolved. He was no longer writing services; he was an orchestrator and reviewer. Because autonomous agents could generate code ten times faster than humans could write it, the bottleneck shifted entirely to verification. To keep the pipeline moving, the engineering org sliced the review process into hyper-specialized, bite-sized validation queues.
Marcus’s days became an endless stream of contextless diffs.
- 9:14 AM: Approve an agent-generated optimization to a connection pooler. The unit tests pass; the linter is happy. Click Approve.
- 9:22 AM: Validate an automated schema migration for a telemetry ingestion service. The agent’s summary says it reduces read latency by 12 percent. Click Approve.
- 9:35 AM: Sign off on a dynamic policy rule drafted by a security agent to auto-remediate anomalous traffic. Click Approve.
Marcus was officially “in the loop.” His digital signature was on the audit trail. If compliance asked whether a human had supervised the AI, the answer was an unequivocal yes.
Except it was a lie.
You cannot meaningfully audit in five minutes what you did not struggle to design over five days. Deprived of the tactile context he used to gain from building things by hand, Marcus was no longer evaluating the systemic truth of the code; he was evaluating whether the code looked plausible. He was sorting scary numbers into bins.
Look at the psychological toll of this shift. We took a brilliant systems thinker who loved solving deep puzzles and turned him into a biological quality-assurance filter — a rubber-stamper sitting at the end of an algorithmic assembly line, absorbing the legal and operational liability for work produced by a machine. He was exhausted at the end of every day, yet he felt a hollow, gnawing detachment. When a system launched, he felt no pride, because none of his scars, experiments, or late-night breakthroughs were inside it.
He was severed from his own craft. And yet, the machine kept building.
Stage III: The Cube Phase (The Headless Labyrinth)
In Vincenzo Natali’s 1997 cult sci-fi thriller Cube, six strangers wake up inside a massive, shifting structure of interlocking cubic rooms, many of which are rigged with lethal, hair-trigger booby traps. For the first half of the film, the characters — and the audience — assume they have been placed there by a sadistic mastermind, a shadowy government cabal, or aliens conducting a twisted behavioral experiment.
The true horror of the film arrives when one of the prisoners, a cynical, beaten-down office worker named Worth, confesses that he worked on the Cube.
When the others pin him against the wall and demand to know who runs the project and what its grand purpose is, Worth delivers a revelation far more terrifying than any conspiracy: nobody runs it, and there is no purpose.
Worth didn’t design a murder maze; he was an architect contracted to calculate the structural specifications for a hollow outer shell of specific dimensions. He didn’t ask what went inside it. Another contractor in another city designed the hydraulic hatches. A metallurgist somewhere else specced the alloys. A systems programmer wrote the permutation routines that shifted the rooms. Nobody held the master blueprint. The project was simply a sprawling chain of modular, siloed contracts propelled forward by bureaucratic inertia. Everyone did their isolated job, optimized their local metrics, cashed their checks, and went home — until the machine was finished, and having no other use for it, the system began putting people inside it.
This is Stage Three. This is what happens when WALL-E’s loss of friction meets Severance’s loss of context inside a complex technological infrastructure.
For Marcus, the awakening came on a Tuesday afternoon in October.
It started as a minor latency spike in an edge routing cluster. Within four minutes, it cascaded into a total systemic lockup across three availability zones. Customer traffic dropped to zero. Internal dashboards went dark — not because the servers were dead, but because the observability pipeline itself was choking.
Marcus opened his terminal, cracked his knuckles, and prepared to be the hero he used to be. He SSH’d into the bastion layer to manually drain the corrupted queues and force a state rollback.
The system rejected his credentials. Automated Security Policy 409-B: Anomalous manual intervention detected during active auto-remediation window. Access revoked.
Sweating, Marcus pulled up the source repository for the security policy to bypass the lock. He looked at the git blame on the exact rule blocking him from saving the cluster.
Approved by: Marcus. January 14. It was an agent-generated hardening patch he had rubber-stamped between meetings because the summary looked clean.
He pivoted to the orchestration layer to kill the runaway retry loops that were DDOSing the internal databases. The retry logic wasn’t using standard exponential backoff; it was using an aggressive, dynamic token-bucket algorithm that was currently starving the control plane of compute cycles. He checked the commit history.
Approved by: Marcus. March 3. A performance optimization ticket generated by an autonomous tuning agent. All unit tests had passed.
Desperate, he tried to pull the raw kernel logs from the storage nodes to see what had triggered the initial fault. The logs weren’t there. To save 18 percent on cloud storage costs, an automated data-lifecycle microservice had classified verbose error payloads during high-throughput events as “redundant telemetry” and purged them at the edge.
Approved by: Marcus. Six days ago.
Marcus sat frozen in the glow of his monitors. The incident bridge was screaming in his headset — executives demanding ETAs, product managers asking who was in control, junior engineers pasting hallucinated AI troubleshooting steps into the chat channel.
And in that moment, the cold, sickening truth hit him.
Nobody was in control. There was no architect anymore. Over two years of chasing frictionless velocity, he and fifty other engineers had validated tens of thousands of hyper-optimized, locally rational, completely siloed fragments of automation. Every single pull request had made sense in isolation. Every single ticket had satisfied its local KPI.
Together, without a single human holding an end-to-end mental model of how those pieces interacted under stress, they had assembled a headless, self-locking labyrinth. Marcus wasn’t the overseer standing outside the machine with a clipboard. He was locked inside the Cube, staring at the hatches, locks, and razor wire that bore his own cryptographic signature.
Stop Looking at the Screen. Look at the Room Around You.
It is tempting to read Marcus’s story as a cautionary fable about five years from now.
Do not flatter yourself. Look at your browser tabs right now.
Think about the last three pull requests, infrastructure changes, or automated workflows you approved this week. Be brutally honest with yourself: did you trace the state transitions from the bare metal to the user interface? Do you know exactly how that generated code behaves when the network partitions, when the disk fills up, or when an upstream dependency starts returning malformed headers?
Or did you read the four-bullet-point AI summary, glance at the green checkmarks from the automated test suite — a test suite that was also written by an agent — and click Approve because you had twelve more tickets in your queue and a standup in ten minutes?
Look around your organization. Who actually holds the master blueprint of your production environment today? Not the idealized boxes-and-arrows diagram on a slide deck from two years ago — the actual, living topology of your code, your infrastructure, your dependencies, and your automated feedback loops.
If the honest answer is “nobody,” then you are already inside the Cube.
We are living through a massive, industry-wide delusion. We have convinced ourselves that we can strip away the slow, painful friction of building (WALL-E), slice human oversight into contextless, high-velocity validation queues (Severance), and somehow avoid waking up inside an unsteerable, emergent catastrophe (Cube).
We are trading legibility for velocity. We are trading sovereignty for convenience. And every time we rubber-stamp a piece of automation we do not fundamentally comprehend, we weld another hatch shut above our heads.
How to Break the Cube: A Manifesto for Architectural Sovereignty
We do not need to smash the machines or return to writing assembly by hand on CRT monitors. Automation is not the enemy. Blindness is the enemy.
If we want to remain the architects of our systems rather than their prisoners, we have to stage a deliberate, disciplined rebellion against how we build and govern technology. That requires three immediate commitments:
1. Reclaim the Friction of the Critical Path
Stop treating all engineering struggle as waste to be optimized away. Yes, let agents scaffold the boilerplate, format the schemas, and draft the mundane utilities. But when it comes to the core state machines, the failure-recovery paths, the security boundaries, and the foundational architecture, keep your hands on the metal. Struggle with the code. Debug it manually. Force your brain to walk the territory so that when the lights go out at 3:00 AM, you possess a living, intuitive map of the room you are standing in. If an optimization robs your team of the ability to understand how the system fails, reject the optimization.
2. Shatter the Severed Silos
Refuse to play the role of the Lumon refiner. Never sign your name to a diff, a configuration change, or an automated workflow in a vacuum. If your organization’s process reduces you to checking syntax and local test coverage without giving you the time and visibility to inspect how a component alters the global system, halt the line. “Human-in-the-loop” means nothing if the loop is a microscopic circle disconnected from the architecture. Demand end-to-end context before you grant your validation.
3. Audit the Program from Every Level
Treat autonomous generation and agentic workflows as untrusted foreign code until proven legible. Institute ruthless, top-to-bottom architectural audits where no AI summary is permitted in the room. Pick a critical service, gather the engineers, and trace a single packet or transaction from the hardware interface through every middleware layer, policy engine, and retry loop, all the way to persistent storage.
If you hit a black box — a module or an automated rule where everyone in the room shrugs and says, “The agent generated that a few sprints ago, and the tests passed” — stop everything. Tear that box open. Read it line by line. And if nobody on the team can explain why it works or reconstruct its logic on a whiteboard, rip it out and rewrite it until someone owns it.
The seduction of the Cube is that it requires no villain to build. It only requires talented, ambitious people to keep their heads down, trust the process, optimize their local squares, and stop asking what the whole structure is doing.
Step out of the floating chair. Refuse to sort the numbers blindly. Pick up a wrench, open the hatches, and audit the machine — before the doors lock from the outside.