When AI Makes Judgment Optional

Simon Carver

[warmly] Welcome to the show. This is The Human Workforce Podcast, and I am genuinely glad you're here. Lachlan, I want to start with a question that feels small when you first hear it, and then... not small at all: what happens when we get so used to asking a machine to think, choose, draft, sort, plan, and even create for us that handing over judgment starts to feel normal?

Lachlan Reed

[reflective] Yeah. That's the bit that gets me. We talk about AI like it's a teammate -- and sometimes it really is. It can knock out the donkey work, the repetitive stuff, the jobs people hate. That's right there in The Last Job You'll Ever Hate: automation can free us from meaningless work. Fair enough. But if we're not careful, mate, convenience sneaks over the line into dependence, and suddenly human judgment feels like an optional extra... like heated seats in a ute.

Simon Carver

[curious] That phrase, "meaningless work," is doing a lot of heavy lifting. Because nobody misses pointless admin for its own sake. But structure? Ritual? The feeling that your choices matter? Those aren't meaningless at all. I keep thinking about how work, for better or worse, has given people a shape to the day. A reason to get up, solve a problem, make a judgment call, be needed by somebody. If the machine takes the task, do we also let it take the needing?

Lachlan Reed

[matter-of-fact] That's it. We've always built tools to extend what humans can do. Hammer, spreadsheet, software, whatever. The healthy version is: the tool helps me do MY job better. The dodgy version is: the tool does the job and I just watch the screen like a stunned mullet, ticking the approve box. And once that happens enough times, your own instincts get rusty. Same as not riding a bike for years -- first time back, even a kangaroo could trip over it.

Simon Carver

[softly] Rusty is the right word. Because this isn't just about employment. It's about agency. If AI writes the email, summarizes the meeting, recommends the hire, drafts the strategy, makes the diagnosis, composes the music -- then purpose starts to become optional. Decorative, almost. Like, sure, a human can add a flourish at the end if they'd like. But the core engine no longer needs them.

Lachlan Reed

[questioning tone] And when purpose becomes optional, who actually keeps choosing it? That's my worry. Most people aren't lazy, but we're all busy, we're all knackered, and we all take the shortcut when the shortcut works. If AI keeps being faster and smoother, we'll use it for the boring bits first. Then the tricky bits. Then the meaningful bits without noticing. Bit by bit, the parts of work that gave us shape -- problem-solving, creating, deciding -- get handed off because, well, the machine's quicker.

Simon Carver

[hesitates] Let me try to say that back, because I think this is the hinge. You're not saying drudgery is noble. You're saying friction sometimes carries meaning. The effort itself can be part of identity. Not all struggle is beautiful -- some of it is just awful -- but some struggle is how we discover what we care about, what we're good at, what we're willing to stand behind.

Lachlan Reed

[responds quickly] Exactly. I'm not romanticising misery. No one wants to spend half their life on pointless forms and password resets. But if you remove ALL friction, you don't just make life easier. You can flatten it. You can turn a person from an active participant into a passenger. And passengers, generally speaking, don't steer.

Simon Carver

[reflective] There's a quiet seduction in that, isn't there? Because "let the system handle it" sounds like relief. It sounds efficient, modern, responsible even. But over time it might also mean fewer moments where a person has to wrestle with uncertainty, make a call, or live with the consequences of that call. And I don't know that you can outsource those things without outsourcing part of being human.

Lachlan Reed

[calm] Yeah. AI as teammate? Beauty. AI as replacement for the human need to matter? That's where the ground gets a bit thin under your boots.

Simon Carver

[urgent] And here's where it gets even harder. This isn't just a story about one company over-automating. It's a race dynamic. The source text uses the phrase "competence trap," and I think that's one of the stickiest phrases in this whole conversation. The trap is simple: no company, no industry, no country feels able to slow down, because the second they pause, somebody else keeps going.

Lachlan Reed

[sharp] Yeah -- competence trap. That's the one to remember. It's not that everyone thinks the race is wise. It's that nobody wants to be the mug who stops running first. If your competitor uses AI to cut costs, speed up hiring, ship code, write campaigns, whatever, you feel pressure to do the same. Not because you've sorted the ethics. Because you don't wanna get smoked in the market.

Simon Carver

[skeptical] But this is where I want to push you a little. When people say, "We can't slow down," I hear an excuse smuggled in as realism. Humans made markets. Humans make laws. Humans set standards. So when we talk like acceleration is weather -- like a storm rolling in -- aren't we dodging responsibility?

Lachlan Reed

[sighs] I get that. But I reckon you're underestimating how brutal the incentives are. In theory, yes, humans can set rules. In practice, governance runs on months and years, and AI development runs on seconds, days, and release cycles. That's the "speed gap." Human institutions are basically trying to police trail bikes with a town-planning committee. By the time the meeting starts, the bike's halfway up the hill.

Simon Carver

[leans in][curious] The "months and years" versus "seconds" part -- that's the detail that lands for me. Because it explains why control can feel real long after it has become symbolic. We still say things like safety layers, oversight boards, kill switches. Those words are comforting. They sound like a hand on the wheel. But if the system is evolving faster than the people supervising it can understand it, what exactly is that hand gripping?

Lachlan Reed

[matter-of-fact] Maybe nothing. Or not enough. That's the nasty bit. We also hit a complexity ceiling: we no longer fully understand the systems we build. Not line by line, not behaviour by behaviour. So we've got the competence trap pushing us forward, the speed gap making response sluggish, and the complexity ceiling making confidence look a lot more solid than it is. That's the illusion of control right there.

Simon Carver

[softly] So the dashboard lights are on, the car is accelerating, and everyone in the front seat is saying, "No, no, it's fine, we've got procedures." [short pause] That is not a great feeling.

Lachlan Reed

[chuckles] Nah, not ideal. Proper white-knuckle stuff. And this is where I think we disagree a bit. You still think a meaningful global slow-down is possible. I think once the race starts, getting broad discipline is bloody hard. Not impossible... just hard enough that "we should slow down" can become a nice sentence people say while flooring it anyway.

Simon Carver

[firm but warm] I do think it's possible -- not easy, but possible. And maybe the better word is not "stop." Maybe it's "refuse." Refuse certain deployments until safety is proven, not assumed. Refuse designs that replace human judgment where human judgment is the whole point. Refuse the childish idea that every capability must be built simply because it can be built.

Lachlan Reed

[reflective] I can get behind "refuse." That's more concrete. Mandatory latency -- slowing deployment. Human-centered design -- AI as augmentation, not substitution. And a bit of moral backbone about saying no to some capabilities. That's not anti-tech. That's just grown-up behaviour. Same as not giving a teenager a faster bike just because the engine fits.

Simon Carver

[warmly] Right. Because the smartest move available to us may not be acceleration. It may be restraint. And restraint doesn't mean fear of the future. Sometimes it means love for what is fragile in the present -- our judgment, our relationships, our sense of purpose, our very human need to matter to the world we are making.

Lachlan Reed

[serious] Yeah. We are the ones building this. Which means if it starts looking like an exit sign for humanity, painted by our own hand, we don't get to act surprised later. [pauses] Anyway... that's the question to sit with: if AI can do more and more, what are the parts of being human we simply refuse to hand over?

Simon Carver

[softly] That's where we'll leave it. Glad you were here with us.