The First Real AI That Works Against Humans

Simon Carver

Welcome back to The Human Workforce Podcast. Today we’re talking about something most people have experienced, but not many people really understand. The first real AI system that doesn’t just assist humans... it works against them.

Lachlan Reed

Yeah. And we’re not talking about killer robots or some sci-fi carry-on. We’re talking about the thing that decides whether you even get a chance to speak. Before the interview. Before a callback. Sometimes before a human being even knows your name.

Simon Carver

Applicant Tracking Systems. ATS. That term sounds boring on purpose. It sounds administrative, harmless, almost helpful. And in the beginning, that was more or less true. Back in the 1990s, these systems were basically digital filing cabinets. A company got a pile of resumes, scanned them in, stored them, organized them, searched them later. Messy? Sure. But the core job was storage.

Lachlan Reed

Right, like a glorified folder drawer. Bit less dusty, bit more searchable. Instead of Janet in HR flipping through manila folders, you had software doing the sorting. Fair enough. No one hears “resume database” and thinks, “Ah yes, the machine that’ll quietly ruin my chances.”

Simon Carver

Exactly. But that’s the pivot. What started as storage became screening. Then ranking. Then elimination. Platforms like Workday, Taleo, Greenhouse, and others didn’t stay passive. They evolved into systems that score, sort, and prioritize candidates automatically.

Lachlan Reed

And that’s where the filing cabinet became a filter. Then the filter became a gatekeeper. These systems now scan resumes for keywords, match job descriptions against phrases, infer skills from job titles and wording, and in plenty of cases use AI or natural language processing to predict fit. Sounds clever. Sometimes is clever. But clever isn’t the same as fair.

Simon Carver

And it isn’t the same as wise. Because pattern recognition has a blind spot: it confuses familiarity with quality. If you’ve done the expected job, used the expected terms, followed the expected path, you look legible to the system. If your background is unusual, if you changed careers, if your best skills don’t line up neatly with the template, you can disappear.

Lachlan Reed

Yeah, and that’s the sting. The system isn’t asking, “Could this person do the job?” It’s asking, “Does this person resemble what we already understand?” Bit like judging a trail bike by the paint instead of whether the engine starts. Terrible way to buy a bike. Pretty ordinary way to hire too.

Simon Carver

And because the process feels technical, people assume it must also be objective. That’s one of the tricks here. The interface looks clean. The workflow looks systematic. The rankings look precise. But precision is not the same thing as truth. A candidate can be ranked low for missing words they’d never use, even if they have years of directly relevant experience.

Lachlan Reed

Or they get bumped because the system likes one phrase over another. “Managed” versus “led.” “Customer success” versus “account management.” Same dog, different collar. But if the machine’s trained to sniff one scent, the other one gets tossed off the ute.

Simon Carver

And what gets lost in all this is the original point of hiring. Hiring used to involve judgment early. Imperfect human judgment, yes, but still judgment. Somebody could look at a resume and think, “This person took a strange path, but there’s something here.” The new model often never gets to that moment.

Lachlan Reed

That’s the key. We didn’t just digitize recruiting. We automated the first rejection. And once that became normal, companies started treating it like a feature instead of a warning sign.

Simon Carver

So when we say this is the first real AI that works against humans, we mean something very specific. Not that software is evil. Not that every recruiter is lazy. We mean the system has been built to serve corporate priorities first: speed, scale, tidiness, risk reduction. Human potential comes second, if it shows up at all.

Lachlan Reed

And if you’ve ever thought, “How on earth did I get rejected that fast?” mate, you’re not imagining it. There’s a good chance no person weighed you up. You were processed. That’s not the same thing. Not even close.

Simon Carver

Let’s make this concrete, because this can sound abstract until it lands in someone’s actual life. There was a time when hiring managers read resumes. Not every one perfectly, not always thoughtfully, but context had a chance. Career pivots mattered. Unconventional experience mattered. Potential mattered. And often, conversation came earlier.

Lachlan Reed

Now? [frustrated] You can be punted in seconds. Thousands of applications come in, the system scans them, stacks them, bins them. And loads of candidates are filtered out before a hiring manager ever sees a single line. That’s not “the market speaking.” That’s a screening workflow chopping the pile down.

Simon Carver

And the volume problem is real. Big applicant pools, more dissatisfaction at work, more people applying broadly, AI-driven layoffs pushing experienced people back into the market. So companies face a flood. But here’s the issue: ATS systems are not really finding the best candidate. They are reducing the pile to something manageable.

Lachlan Reed

Yeah. Important distinction. It’s triage, not discovery. And when the goal is reduction, good people get dropped all the time. The machine isn’t saying, “This person lacks potential.” It’s saying, “This person doesn’t match the pattern tightly enough.” That’s a totally different call.

Simon Carver

CJ Murphy’s experience brings this into focus. Over more than 20 years, he applied to hundreds of roles. Again and again, instant rejection emails. Roles where he met the qualifications. Roles where, on paper, he should at least have reached a first conversation. But no conversation came.

Lachlan Reed

And this bit really gets under your skin. He applied to the same company twice within a month. Same basic candidate, same capability. The big change was he lowered his salary expectation significantly. Suddenly, he got a call from HR.

Simon Carver

And during that interaction, it became clear he was highly qualified. So the issue wasn’t capability. It wasn’t relevance. It wasn’t that he couldn’t do the work. The compensation expectation was acting like a tripwire.

Lachlan Reed

That’s not hiring. That’s filtering on a cost constraint the manager may never even see. Somewhere in that process, the question stopped being, “Can this person help us?” and became, “Can we get this person cheap enough?” Different sport entirely.

Simon Carver

And then came the generic message: better candidates. That phrase does a lot of damage. Because it sounds comparative and thoughtful. But in cases like this, it may just mean the system or the workflow found someone who fit the budget profile better.

Lachlan Reed

Or who typed the right magic beans into the form. Let’s talk salary fields for a sec. A lot of applications require compensation expectations up front. Companies frame that as efficiency. Sometimes maybe it is. But it also works as a pre-screen elimination tool. You are not being evaluated for what you can do. You’re being filtered for what you might cost.

Simon Carver

And that changes HR’s role too. Instead of being a bridge between talent and the business, HR can become a gatekeeper of cost control. Managers lose visibility into strong candidates. Candidates lose any chance to explain context. And everyone gets the illusion that the process was merit-based because the rejection arrived from a polished system.

Lachlan Reed

There’s also a deeper ethical mess here. Lack of transparency. No real explanation. Potential bias baked into decision logic. You get excluded, but you don’t know why. You can’t challenge it. You can’t correct it. You can’t even tell if the issue was your background, your wording, your salary number, or just bad luck.

Simon Carver

And psychologically, that matters. Repeated automated rejection wears people down. The first experience someone has with your company might be a machine saying no. No feedback. No human acknowledgment. Just a cold little note telling them to keep trying somewhere else.

Lachlan Reed

Which is rough, because people aren’t inventory. They’re not barcodes in a warehouse. They’ve got rent, families, pride, hopes, all the rest of it. But the workflow treats them like a spreadsheet row. Tick, flick, reject. Brutal stuff.

Simon Carver

So what is this system actually optimizing for? Not talent discovery, not really. It optimizes for speed, cost control, and risk avoidance. Get through the pile fast. Keep compensation predictable. Reduce perceived uncertainty by favoring familiar patterns.

Lachlan Reed

And that sounds tidy in a boardroom, but it comes at a cost. You lose diversity of experience. You miss unconventional talent. You stop building long-term capability because you’re chasing the safest-looking match. It’s like only planting the same crop every season because it’s easy to measure. Sooner or later the soil goes crook.

Simon Carver

This is also part of a bigger pattern we keep talking about. AI isn’t only being used to automate tasks. It’s being used to justify workforce reduction, to narrow access, to ignore internal talent, to prioritize external signals over actual human development. And when the front door is filtered this aggressively, it becomes much easier to favor people already inside the circle.

Lachlan Reed

Yeah — if the system blocks qualified people from entering, it’s a lot easier to hire who you already know, or who already looks familiar, or who sits closer to power. Merit gets talked about. Proximity wins. Same old story, flashier dashboard.

Simon Carver

That’s why the title of this episode matters. This isn’t AI replacing jobs in some distant future. This is AI deciding who never gets one. Right now. Quietly. At scale.

Lachlan Reed

And once hiring becomes purely transactional, dignity cops a hiding. People become data points. The company says it’s being efficient. The candidate feels invisible. No feedback loop. No conversation. No acknowledgment that a career is more than keyword density and a salary box.

Simon Carver

So what do we do? For companies, first: reintroduce human checkpoints early. Not at the very end after the machine has done all the damage. Early. Second: remove rigid compensation filters where possible. If pay matters, fine, but don’t let that be an invisible trapdoor. Third: audit these systems for bias and over-filtering. And finally, treat hiring as an investment decision, not just a cost-control exercise.

Lachlan Reed

For candidates, you’ve gotta be practical. Optimize your resume for ATS because, whether we like it or not, that’s the game on the field. Use the language that maps to the role. Spell out measurable business outcomes. Don’t assume the system will infer your brilliance from vibe alone. It won’t. Even a kangaroo could trip over that one.

Simon Carver

But don’t rely on the system either. Build network-based entry points. Referrals, direct outreach, real conversations, communities, former colleagues. Anything that helps you reach a human being before the software reduces you to a score.

Lachlan Reed

And understand this clearly: if the process feels like it’s working against you, in many cases it is. That doesn’t mean give up. It means stop personalizing what is often structural. Adjust the strategy. Don’t let a machine tell you your value.

Simon Carver

The first real AI that works against humans isn’t coming. It’s already here, and it’s deciding who gets a chance.

Lachlan Reed

That’s the episode. Bit of a hard one, but worth saying straight. See you next time, Simon.

Simon Carver

See you next time, Lachlan. Take care, everybody.