From Chatbot Mistakes to AI Agency Risk

Simon Carver

Most business leaders think that putting a little disclaimer under an artificial intelligence tool completely shields them if it gives out bad advice. But, er, back in November 2022, a guy named Jake Moffatt was buying a plane ticket after his grandmother passed away. He asked the Air Canada chatbot about bereavement fares, and the bot told him, you know, buy the full price ticket now and apply for the discount later. Turns out that was totally against actual company policy.

Lachlan Reed

November 2022! And when Jake asked for his refund, Air Canada basically said, oh, er, the chatbot made a mistake, that is not our policy, so we are not paying.

Simon Carver

Right! They literally argued in front of British Columbia's Civil Resolution Tribunal that the chatbot was essentially its own separate legal entity and that the company, um, should not be held responsible for what its software said.

Lachlan Reed

Mate, that is like me sitting in my backyard shed, working on an old trail bike, leaving the brake line completely unbolted, sticking a little paper warning sign on the handlebars saying, caution, brakes might fail, and then being shocked when I crash into the back fence! You cannot just slap a disclaimer on something you built and own and pretend you had no part in the damage.

Simon Carver

And the tribunal completely shot Air Canada down. They said, look, if a customer gets information from your website, whether it comes from a static page or an interactive bot, it is still the company talking. But, er, here is where this gets really interesting. That Air Canada bot could only talk. Now we are deploying systems that actually reveal how AI agents are transforming the way professionals, entrepreneurs, and organizations work.

Lachlan Reed

That is the massive jump, right? We went from chatbots that just answer questions to autonomous enterprise agents that can look up customer accounts, rewrite tickets, issue actual money refunds, and click buttons inside internal workflows. You are moving from generative risk to full on agency risk.

Simon Carver

Exactly. Think about it in plain language terms. If a human customer service employee misinterprets company policy and gives away a five hundred dollar refund by mistake, who pays for that? The company does! You cannot go to court and say, well, er, section four of our handbook says otherwise, so our employee's mistake does not count. Swapping out a human staff member for an algorithmic agent changes how fast the task gets executed, but it does not magically wipe away legal responsibility.

Lachlan Reed

Spot on. If it is your agent operating inside your architecture, doing tasks on your behalf, you own the outcome. End of story.

Simon Carver

And courts are doubling down on this human accountability angle everywhere. Look at Mata versus Avianca, or that big 2025 litigation involving Walmart, where lawyers submitted legal briefs containing completely fictitious case citations generated by artificial intelligence. The federal judges did not blame the software. They sanctioned the lawyers directly!

Lachlan Reed

Mata versus Avianca was wild! Those judges made it crystal clear. Automation does not eliminate professional liability, it just concentrates it right on the person who clicked submit. But, er, what happens when the agent itself gets confused because the internal corporate data is a complete mess?

Simon Carver

Oh, the information truth paradox! This happens all the time. An agent queries a system, and SharePoint has an old 2021 procedure PDF saying one thing, while the main intranet page says another. The agent pulls the wrong document and executes a decision based on outdated rules. We saw this exact mess with New York City's official business advice chatbot, which was caught giving local shop owners advice that directly contradicted actual city regulations and housing codes!

Lachlan Reed

Yeah, it had disclaimers everywhere, but business owners were following official government portal guidance! Which proves that artificial intelligence governance is totally useless if you do not have strict data governance first. If your internal sources of truth are conflicting, your autonomous agent is basically just automated chaos.

Simon Carver

And regulators are catching up fast. Under the European Union AI Act, major transparency rules kicked in on August 2, 2026, forcing companies to clearly document risk, oversight, and deployment responsibilities. You can no longer treat software as an autonomous legal scapegoat.

Lachlan Reed

And the market is already reacting. Gartner released a massive prediction warning that over 40 percent of agentic artificial intelligence deployments will be completely canceled or decommissioned by the end of 2027. And why? Because of governance gaps discovered after major production incidents!

Simon Carver

40 percent! That line, after production incidents, should terrify any corporate executive. It means companies are launching these autonomous agents without proper guardrails, watching them make a costly mistake in the wild, and then scrambling to pull the plug because they realized they built up massive unhedged risk.

Lachlan Reed

So the real executive question is not, er, how smart is our tool? It is, what authority did we hand over, who verifies its decisions, and who carries the can when it fails? You can delegate the work, but you can never delegate the legal responsibility.