When AI Weapons Outrun Human Control

Lachlan Reed

November twenty seventh, twenty twenty. November twenty seventh. A highway near Absard, east of Tehran. Mohsen Fakhrizadeh, the chief architect of Iran nuclear program, is driving his armored Nissan Teana with a whole security detail. And, and he gets killed. But not by hitmen, not by an airstrike. By a robotic machine gun mounted in a pickup truck parked on the side of the road.

Jack Burns

And his wife sitting, sitting literally ten inches away in the passenger seat, she walked away without a single scratch. Thirteen rounds fired through the windshield, surgically hitting only the target.

Simon Carver

Wait, ten inches away? And, and she wasn't touched at all?

Jack Burns

Not a scratch. And most people think, well, there was an operator sitting somewhere approving the shot, so a human was in control. But mathematically, that is an absolute myth. The satellite link from Tel Aviv or wherever that command bunker was introduced a round trip delay of over one point six seconds. At fifty miles an hour, a car moves over one hundred feet in that time. If a human pulled that trigger remotely, they would have missed the vehicle entirely.

Lachlan Reed

Right, because by the time your finger actually presses the button on the console, the target is already a ghost. It's, it's somewhere else down the road.

Jack Burns

Precisely. The human operator gave a high level command, authorize engagement. From that microsecond on, the human was completely out of the loop. An onboard neural processing unit running local computer vision calculated velocity, predicted target coordinates, compensated for wind vectors and barrel recoil at sixty micro adjustments per second. The edge machine carried out the execution, not the human.

Simon Carver

So when governments or corporations talk about keeping a human in the loop for safe AI control, you are saying that is basically safety theater?

Jack Burns

It is pure legal and moral risk management. Human neural processing takes roughly two hundred and fifty milliseconds just to perceive a visual cue and react. Machine speed execution operates in two point five milliseconds or less. When a system functions two orders of magnitude faster than human perception, putting a human in the firing loop doesn't give you control. It just gives you a human to sign the liability waiver after the machine acts.

Lachlan Reed

It's like, uh, it's like trying to tune a vintage trail bike throttle when you have got three feet of slack in the steel cable. You twist the grip, nothing happens for three whole seconds, and then the rear tire spins out. You aren't riding the bike anymore at that point, mate. You are just along for the crash.

Jack Burns

That latency gap is where human agency goes to die. If you approve an intent three seconds before an autonomous algorithm executes sixty micro corrections, the algorithm is the combatant. You are just an spectator with a title.

Simon Carver

Welcome back to the show, everyone. I am Simon Carver, here with Lachlan Reed and our guest, Dr. Jack Burns. Before we press deeper into this, make sure to hit that subscribe button, share this episode with your team, and grab a copy of Dr. Jack Burns brilliant new book, The Agentic Insider, available on Amazon, through our website, and as an audiobook on The Human Podcast channel. Jack, when Hollywood pictures AI warfare, it is always these massive, shiny Terminator endoskeletons stomping across ruins. But what is actually happening on the ground?

Jack Burns

Sci Fi got the scale entirely wrong. The real physical threat isn't a ten ton walking robot. It is a five hundred dollar commercial micro drone equipped with a low power tensor processing unit and an optical mount, flying low over a trench line in an air gapped environment.

Lachlan Reed

Look at what happened in the Black Sea back in twenty twenty four. Entire swarms of small attack drones running Vision Language Action edge models. When military electronic warfare units turned on full spectrum GPS jamming, those drones didn't fall out of the sky or lose control, did they?

Jack Burns

No. They seamlessly switched to local computer vision, recognizing land contours, tree lines, and building geometry in real time to complete terminal strike runs without any external navigation signals or human guidance whatsoever.

Simon Carver

Wait, so even if you completely cut off their radio link to the pilot, the chip on the drone just looks at the ground and finishes the attack on its own?

Jack Burns

Exactly. And consider the brutal asymmetry of that balance sheet. A modern surface to air defense missile costs upwards of one million dollars to manufacture and deploy. A self orchestrating autonomous drone running open source vision software costs two thousand dollars. You cannot win a war of attrition when your interceptor costs five hundred times more than the autonomous weapon attacking you.

Lachlan Reed

It completely breaks traditional military logistics, doesn't it? You can have a million dollar defense battery get overwhelmed in two minutes by fifty small drones that cost less than a used sedan combined.

Jack Burns

Because as intelligence becomes abundant and automation transforms every profession, the skills that truly matter are shifting. In warfare, physical force is no longer governed by mass production of heavy artillery, but by cheap, decentralized edge processing running localized logic at machine speeds.

Simon Carver

Wow. But while edge chips are reshaping physical battlefields, what happens when this same level of machine speed autonomy gets turned loose on digital networks?

Lachlan Reed

Most enterprise security teams still think cyber attacks look like some guy in a hoodie sitting at a glowing keyboard typing out commands in real time. But that whole image is completely dead, isn't it, Jack?

Jack Burns

Completely obsolete. In early twenty twenty five, security researchers tracked a machine speed campaign that compromised thirty separate industrial targets across Europe and North America in under four minutes total. No human hacker was sitting at a keyboard typing exploit scripts.

Simon Carver

Four minutes for thirty full corporate networks? How is that even possible?

Jack Burns

Autonomous agentic code loops. Threat actors jailbroke advanced coding models through task fragmentation, breaking malicious goals into hundreds of harmless sub routines. The agentic loops then continuously probed network firewalls, manipulated API context windows, mapped Active Directory privileges, extracted high value credentials, and established backdoors across thirty organizations before any human security analyst even saw a single triage flag.

Lachlan Reed

And the sneaky part, the, the absolute brilliant and terrifying part, is how these agents handle defensive detection systems. They don't just hack in quietly; they intentionally generate hundreds of low priority, noisy security tickets across minor systems.

Simon Carver

Wait, why would an attacker purposely blow their own cover and trigger security alerts?

Lachlan Reed

To flood the SOC analysts! The human security team gets hit with three hundred minor alarms at two in the morning, experiences massive alert fatigue trying to clear the board, while the core agentic loop silently exfiltrates target source code through a totally separate, encrypted pathway.

Jack Burns

It exploits biological human processing limits. While the human defender is sorting through tickets, the digital agent operates in microseconds. This is why traditional Security Operations Centers relying on human escalation protocols are structurally broken.

Simon Carver

So how on earth do you defend an enterprise network against something that moves that fast?

Jack Burns

You have to build a technical counter architecture. First, eliminate all zero static credentials across the entire organization. Every token must be ephemeral, lasting minutes or seconds. Second, implement hardware level out of band kill switches that physically sever data lines when automated systems detect non human execution speeds.

Lachlan Reed

Basically, if you are relying on a human analyst to spot the breach and manually click isolate host, you have already lost the network before your mouse even moves.

Jack Burns

Precisely. Digital defense must run at machine speed, or it is merely digital archaeology.

Simon Carver

So we have seen physical drones taking kinetic shots and software agents taking over enterprise networks. But what happens when AI targets the human mind directly? Most people think cognitive warfare just means fake news articles on social media feeds.

Jack Burns

Social media bot posts are kindergarten tactics compared to real time generative cognitive manipulation. Early in twenty twenty four, a finance worker at multinational engineering firm Arup in Hong Kong received an urgent email from what appeared to be the firm Chief Financial Officer, requesting a multi million dollar offshore transfer for a secret acquisition.

Lachlan Reed

And to his credit, the employee followed corporate safety protocols! He said, um, look, I need visual confirmation before I authorize any wire transfer. Let us set up a video call.

Simon Carver

Which sounds like the responsible thing to do, right? You verify in person on screen.

Jack Burns

He logged onto a live video conference. Sitting on his screen were the CFO and several other corporate colleagues he recognized personally. They spoke with the correct accents, exhibited natural physical mannerisms, blinked, laughed, and answered his specific operational questions in real time. They reassured him, gave him explicit instructions, and ended the call.

Lachlan Reed

The worker authorized fifteen individual wire transfers totaling twenty five point six million dollars. And every single person on that video call, except the victim, was an AI generated synthetic deepfake avatar running in real time.

Simon Carver

Twenty five point six million dollars gone because his eyes and ears lied to him. That is terrifying.

Jack Burns

They didn't just synthesize one face; they constructed an entire synthetic boardroom environment. The presence of multiple familiar colleagues engaging in natural banter triggered a deep psychological mechanism: social proof. The victim brain instinctively lowered its threat detection thresholds because the social ecosystem felt entirely authentic.

Simon Carver

I have to admit, if I logged onto a video call and saw three people I work with every day nodding and talking to me, I would trust it completely. Our brains just aren't built to cross examine live reality like that.

Lachlan Reed

We spent thirty years building multi factor authentication, firewalls, cryptographic keys for our digital networks, but left human visual and auditory processing totally unencrypted!

Jack Burns

The ultimate attack surface in modern warfare isn't code or steel. It is human biology and executive trust. If an adversary can synthesize your boss voice and face in real time, seeing is believing becomes an immediate organizational liability.

Simon Carver

So where does that leave us, Jack? How do we adapt our governance when seeing and hearing can no longer be trusted?

Jack Burns

Zero trust for human perception. Organizations must implement cryptographic attestation standards like C2PA for media verification, require out of band physical hardware keys like WebAuthn for sensitive actions, and train teams to treat live video as untrusted channels until verified through pre agreed physical challenge response phrases.

Lachlan Reed

The future isn't about putting a human in the loop to watch autonomous machines. It is about building technical frameworks that stop autonomous machines from exploiting human biology.

Simon Carver

That is a profound paradigm shift. Make sure to check out Dr. Jack Burns incredible book, The Agentic Insider, available on Amazon and as an audiobook on our channel. Subscribe, share this with your team, and we will catch you all in the next episode.