Last week we ran a small, closed-door roundtable at Leeds Beckett. No panel at the front, no vendors in the room, no slides selling anything. Just a dozen senior people from banking, cyber security, higher education, social care, government, and the gaming and entertainment sector, talking honestly about the real risk of AI adoption, and what AI looks like once you get past the launch announcement.
I expected security to dominate. It barely came up.
What united almost everyone was a question they’d arrived at the hard way. What is this actually for?
From “have we got AI” to “what is AI solving”
Most organisations in the room had already turned the tools on. Often quickly, often under pressure from the top, often out of a fear of being left behind. That part was easy.
Working out which of it mattered was harder, and that was where everyone now lived. The conversation had moved from rollout to value. Where is this genuinely helping, and how would we even know?
The examples that landed were narrow and human. AI is helping a care provider give better guidance. Supporting a penetration tester. Taking marking off a lecturer’s plate. Speeding up research. A multiplier for skilled people who can already judge the output.
That judgment did a lot of work. In skilled hands, AI extends what someone can do. Put it in front of someone who can’t spot a wrong answer and you have a very articulate liability.
The bill nobody can forecast
Cost came up again and again, and not in the way you’d expect. Nobody thought AI was expensive in the abstract. They thought it was unpredictable.
One contributor described rationing tokens to developers. Others talked about walling off the AI that touches customers from the AI people are just experimenting with. Token-based pricing, compute, licensing, and infrastructure. All of it moves.
The worry underneath was sharper than a budget line. Dependency is building faster than understanding. People are leaning on these tools before anyone has worked out what they cost at scale, or who owns that number.
A governance gap, dressed up as a finance question.
AI’s verification tax
Several people described the quiet drain of checking AI’s work. The output looks confident. It reads well. Often it is subtly incomplete or simply wrong.
So the time you saved generating it, you give back to verifying it. And you only catch the errors you were already equipped to spot.
The more content, code and recommendations AI produces, the more an organisation needs a way to validate quality and keep someone accountable for it.
A few people went further. Maybe we should be building friction back in, deliberately, so people keep thinking instead of nodding along to a machine.
Nobody remembers phone numbers any more
Think back to before mobiles. Most of us carried a dozen numbers in our heads. Home, work, your mum, a couple of mates. The phone arrived, the phone remembered them for us, and we stopped. Ask most people for their partner’s number now and they reach for a device.
Same with directions. We follow the blue line on Google Maps and arrive without ever learning the route. Turn the app off and plenty of us are lost in a city we have lived in for years.
No great loss. You can live a full life without memorising numbers or reading a paper map. The skill we are handing over now is a different order of thing. It is thinking. Recall, judgment, the work of reasoning something through, the instinct that tells you an answer is off before you can say why.
Lean on a tool for that long enough and the muscle wastes. People who spent years building their judgement start to lose its edge. That is the smaller risk.
The bigger one sits with whoever is starting out. Reach for AI from day one, and you may never build judgment at all. You cannot erode a skill that was never there. We could end up doing this without meaning to, building workplaces where people never get the chance to develop it. And from the outside, you cannot tell the difference between someone who knows and someone whose tool knows, until the tool is wrong, or gone.
The gap you won’t feel opening
Set the skill problem beside the cost one. The price of these tools will not sit still, and the more you rely on them the less able you are to walk away. Picture the bill climbing on something you can no longer do without, because the ability to do it yourself has quietly wasted.
The bill is then paid in one of two ways. Out of a cash flow that might not stretch, or out of a head that has lost the skill to manage without the tool. Plenty of people will pay with both.
Widen that out and a gap appears. The people and organisations who can afford the best tools, and keep their wits about them, get further ahead. Everyone else slips back, a bit at a time, across business, education and research.
The hard part is how little you feel it happening. There is no alarm. You look up one day further behind than you were, wondering how the distance opened up so fast.
The mess that builds quietly
Because AI makes it easy to build things, people build things. A spreadsheet becomes a web app. Three teams quietly build the same agent. None of it documented, supported or owned.
It works, right up until it doesn’t, and nobody knows how it was put together. The room recognised it for what it was. Technical debt, piling up faster than usual, because the barrier to creating it has dropped to almost nothing.
Banning experimentation kills the upside. The job is to give people somewhere safe to do it, with enough oversight that the useful stuff survives and the rest doesn’t sprawl.
The hard problems were human
Almost none of the hard problems in the room were technical. They were about people.
Confidence and fear. Reliance and lost capability. What does “expert” even mean when a junior with a good prompt can produce a senior-looking answer? Well-being, when change outruns the pace people can absorb it.
The fix is old-fashioned
The shared conclusion was simple. AI adoption is an operating-model problem. People, process, cost, data, security and skills, all at once. The software is the easy bit.
None of this is an argument against AI. Use it. The upside is real, and sitting it out is its own kind of risk. The argument is about going in with your eyes open.
And the way you do that turns out to be old-fashioned. Most of it predates AI by decades.
Start with why. Be honest about where you are now before you reach for anything new. Map the process you actually run, not the one on the org chart. Find the real points of value, and be honest about the risks, the dependencies, the assumptions, and where your data ends up.
Then build some human friction back in, on purpose. Partly that protects jobs, which matters. Mostly, it protects the business and keeps your choices open.
Because the ground will move. The price will climb. The infrastructure will bottleneck. A feature you have come to rely on gets pulled by a government on the other side of the world. The businesses that kept a hand on their own process, and a bit of their own judgement, can adapt when it does. The ones who handed the lot over can only pay up or stall.
None of it is glamorous. It is mapping, hard questions and a bit of deliberate friction. It is also the difference between adopting AI and being owned by it.
Common Questions on the Real Risk of AI Adoption
These were the questions the room kept returning to; raised and debated by senior leaders across banking, government, cyber, education, care and gaming.
Is AI adoption mainly a technology problem?
No. In practice, the hardest parts are human and operational: deciding what is worth doing, controlling unpredictable costs, checking the quality of AI output, and protecting the skills people lose when they lean on the tools. The technology is usually the easy part.
What is the verification tax in AI?
It is the time you spend checking AI’s work. The output looks confident and reads well, but it is often subtly wrong, so the time saved generating it is given back to verifying it. You only catch the errors you were already equipped to spot.
Does relying on AI erode skills?
Yes, in two ways. Experienced people slowly lose the edge on judgment they no longer use. People starting out may never build that judgement at all, which is harder to see, because from the outside, you cannot tell the difference between someone who knows their trade and someone whose tool knows it for them.
How should an organisation approach AI adoption?
Start with the problem rather than the tool. Map the process you actually run, find the real points of value and the risks, dependencies and assumptions underneath them, and build some human judgement back into the workflow so you keep the option to change course when cost or technology shifts.
