Partech Systems — Journal

Your review process was built for a world where output was hard

Bhavna Ate — Thu, 13 Aug 2026 00:00:00 +0000

A head of engineering said something to me in the spring that I've been chewing on ever since. He was looking at his team's half-year numbers and he said, almost apologetically, "everyone's a top performer now and I don't believe it."

He wasn't being unkind. The numbers had genuinely gone up — throughput, tickets, merged work, all of it — across the whole team, including the people he'd have told you six months earlier were struggling. His review process was going to hand out uniformly strong ratings, his budget wouldn't cover uniformly strong outcomes, and he had no defensible basis on which to differentiate.

That's the problem in miniature, and it is arriving in a lot of organisations at once. Most performance systems, whatever they claim in the framework document, ultimately reward visible output. That was a reasonable proxy for as long as producing output was the expensive part. It isn't any more, and a proxy that everyone can now satisfy has stopped being a measurement.

The two bad reflexes

I've seen two instinctive responses and I'd argue against both.

The first is to raise the bar quietly. If everyone's doing twice as much, expect twice as much, and carry on with the same scale. This is what happens by default, because it requires no decision and no announcement. It's also how you get a workforce that received a genuinely useful tool and experienced it as an increase in workload with no acknowledgement — which is a fast route to the sort of resentment that doesn't show up in an engagement survey until it's structural. If you are going to reset expectations, reset them once, deliberately, out loud, with the reasoning attached. Drift is the cruellest way to do it.

The second reflex is to start measuring AI use itself. Put "adoption of AI tools" in the objectives, track licence usage, reward the enthusiastic. I understand the impulse — leadership wants to see the investment landing — but you are measuring an input, and the moment you attach a rating to an input you get theatre. People will use the tool for things it isn't good for, and the person who correctly judged that their work didn't need it gets marked down for good judgement. I've already watched this happen at one organisation. Six months of impressive dashboards and no discernible change in anything that mattered.

What's actually left to distinguish people by

If output is now cheap and roughly equal, what's expensive and unequal?

Judgement, mostly. Which is a soft word for a set of quite hard, specific things.

Choosing the right problem. When it takes an afternoon rather than a fortnight to produce a thing, the cost of producing the wrong thing drops too — so more wrong things get produced, and the person who works out what's actually worth doing becomes disproportionately valuable. That has always been true. It's just no longer masked by the effort involved in execution.

Knowing when the answer is wrong. This is the one I'd weight most heavily, and it's the hardest to see. The engineer who noticed that the generated code handled the timezone case incorrectly, the analyst who spotted that the summarised figures had double-counted a region — they prevented something, and prevention is invisible in every performance system I've ever read. If you don't deliberately go looking for it, you will rate the person who shipped fast above the person who stopped them shipping wrong.

Deciding not to. Restraint has never been rewarded in performance reviews and it's more valuable now than it's ever been.

How to evidence that without it becoming a personality contest

Here's my real worry, and I want to name it plainly rather than let it sit under the surface.

When you shift assessment from countable output to judgement, you shift it towards things that are argued for rather than demonstrated. And the people who are good at arguing for their own contribution are not a random sample. They skew towards the confident, the fluent, the socially central, the ones who look like the person doing the assessing. Every organisation I've worked in has had at least one quiet, excellent person whose value was obvious to their team and invisible to the process. Making the criteria more subjective makes that worse, systematically, and it makes it worse for exactly the groups your inclusion data is already telling you about.

So the criteria have to get more subjective and the evidence has to get more concrete, at the same time. That's the needle.

Practically, what I'd change. Ask for decisions, not volume — three things you decided this half, what the alternatives were, and how it turned out. Ask managers to record catches as they happen, in a line or two, in the moment, because nobody reconstructs them at review time. Ask peers a specific question rather than a general one: not "rate this person's collaboration," but "when did this person save you from something?" And separate the two questions that reviews currently conflate — did the organisation get value from this area of work, which is a team-level question, and did this individual work well, which is not the same thing and increasingly isn't even correlated.

One more, and it applies to anyone with a promotion ladder. Most ladders assume that doing the junior work well is how you demonstrate readiness for the senior work. If the junior work is now largely automated, that evidence path has quietly closed, and you will find yourself with a cohort of people who have no way to prove they're ready. That's a design problem in the ladder, not a shortcoming in the people, and it needs fixing before the first promotion round where it bites.

None of this is a small edit to the form. It's a change to what you believe good work is, which is why most organisations will do the quiet-bar-raise instead. I'd just note that the ones who take it seriously will be able to answer the question my engineering friend couldn't, and the ones who don't will spend the next few years handing out ratings they don't believe.

The customer is the daughter, not the mother

Dimple Paratey — Thu, 09 Jul 2026 00:00:00 +0000

My aunt is eighty-three and lives on her own in a terraced house in Leicester that she has no intention of leaving. There is a family WhatsApp group, and a good proportion of what happens in it is her children and nieces quietly worrying about her while being careful not to let her see us doing it.

Last year one of my cousins bought her a sensor system. Little discs on the doors and the kettle and the medicine cupboard, an app on our phones, and a gentle promise that we'd know if something was wrong. It worked exactly as advertised. We could see when she'd got up, when she'd made tea, whether she'd opened the pill drawer.

She hated it. Not loudly — she's not a loud person — but within about six weeks the discs had been "knocked off by the cleaner," all of them, in an impressive coincidence.

I've thought about that a great deal since, because it taught me something I now see everywhere in this market. The product worked. It just wasn't for her.

Who's actually paying

Almost every piece of AI-adjacent technology sold into ageing is bought by the adult child and used by the parent. The buyer's problem is anxiety. The user's problem is entirely different — she isn't anxious, she's fine, and what she wants is for everyone to stop treating her like a project.

Once you see this you can't unsee it, and it explains why so much of the category feels slightly off. The features are optimised for the reassurance of the person holding the phone: dashboards, streaks, alerts, activity summaries. The person in the house gets no interface at all. She's not a user of that product. She's the data source.

Which is a very strange thing to do to an adult who ran a school office for thirty years and does the cryptic crossword faster than I do.

I'm not against remote monitoring — falls are genuinely how independence ends for a lot of people, and a system that gets someone found in twenty minutes instead of nine hours is not a gimmick. But there's an enormous difference between "this alerts someone if I fall" and "this reports my day to my children," and the products blur it deliberately, because the blur is what sells.

What she does use

Here's the interesting part. She's not a technophobe. She uses things constantly — they're just the ones where she's the one being helped.

She photographs letters and has them read out to her, because the print on the pension correspondence is absurd and her eyes are tired by four o'clock. She dictates messages rather than typing them, at length, with punctuation spoken aloud in a way I find delightful. After a hospital appointment last winter she recorded the consultation, with permission, and got a plain summary of it afterwards — which mattered because she'd taken in about a third of what the registrar said, as most people do in that chair.

None of that is marketed to older people. There's no "silver" branding on any of it. It's just ordinary technology that happens to remove a specific, concrete obstacle, and she found most of it herself.

The pattern seems to be: things that restore capability get adopted, things that transfer oversight to someone else get resisted. That's not a subtle distinction and yet the sector keeps building the second kind.

The companion question

I'll be honest that I don't know what I think about the companionship products, and I'd distrust anyone who's certain.

Loneliness in older people is a genuine health crisis, not a soft issue — the physiological effects are on par with things we take extremely seriously. A device that talks to someone every day, remembers what they said yesterday, asks about the grandchildren by name, is measurably better than the silence it replaces. I've read the research and I've spoken to people who work in care homes, and they're mostly not sentimental about it. If it helps, it helps.

What I can't get past is the direction it lets things drift. It's very easy to imagine a commissioning decision where a companion product is funded because it's cheaper than the visiting service it quietly displaces, and nobody ever writes down that trade because nobody ever proposes it out loud. That's how it would happen. Not a decision — a substitution, made on a spreadsheet.

So my position, for now, is: fine as an addition, never as a saving. And I'd want the person it's for to have been asked, in a real conversation, whether they want it.

The test I'd apply

If you're building in this space, one question. Can she turn it off?

Not "is there a setting." Can the eighty-three-year-old woman in the house decide, without a phone call to her family, without a fuss, without anyone being informed, that today she'd rather not be observed — and does the product still work for her afterwards?

If yes, you've built something for her. If no, you've built something for us, and you should at least be honest in the marketing about which one it is.

My aunt is fine, by the way. She's had a wrist alarm since the spring, which she chose herself, in the shop, having asked several sharp questions about who gets called first. That one's still on.

It was fine in March

Bhaskar Paratey — Thu, 28 May 2026 00:00:00 +0000

The sentence I hear most often when something has gone wrong with a production AI system is: "It was fine in March."

It probably was. That's the problem. Ordinary software fails in a way that sets off a pager — an exception, a timeout, a queue backing up. A system with a model in it degrades instead. The answers get a bit vaguer, the classifications drift a couple of points, the extraction starts missing one field on documents from one supplier. Every dashboard stays green, because every dashboard is measuring whether the thing responded, and it did. It responded beautifully. It was just increasingly wrong.

Then in June somebody escalates a complaint, you go and look, and you find the decline started in April.

Three ways it rots

Worth separating these, because they need different responses.

The inputs change. You built a support classifier on last year's tickets; since then the company launched two products, the help centre was rewritten, and a chunk of traffic moved to mobile where people type shorter and worse. Nothing about the model changed. The world it was fitted to did.

The right answer changes. This one's nastier because the inputs look identical. A fraud pattern evolves specifically to look normal. A policy is updated, so a document that was compliant in January isn't in May, and the model is confidently applying the old rule with the old confidence.

And — by a distance the most common in practice — somebody upstream changed something. A form got a new optional field. An API started returning nulls where it used to return empty strings. A supplier switched their invoice template. A PDF library was upgraded and now the tables extract in a different column order. None of this is an AI problem at all, but it presents as one, and teams lose weeks investigating the model when the culprit was a release note nobody read.

There's also the reflexive case, which I'd file under "know that it exists." If your model's outputs shape the data you later learn from — it ranks what people see, so people click what it ranked — then it's slowly training on its own opinions. That takes longer to hurt you and is much harder to unpick.

What to watch

Not accuracy, mostly, because you rarely have the ground truth in time. Watch the things that move before accuracy does.

Input distribution. Length, language, source system, document type, the share of requests hitting each category. You don't need anything statistically elegant here; a weekly comparison against a fixed reference window catches almost everything worth catching. When the shape of what's arriving changes, something changed upstream, and you want to know that in week one rather than month three.

Output distribution. If a classifier that has always put 12% of tickets in "billing" is putting 19% there this week, either your customers have had an unusual week or you have a problem. Both are worth a look.

Fallback and refusal rates. How often does it decline, hedge, return an empty result, or trip a guardrail? This is a superb early sensor because it moves first and it's trivial to log.

Human override rate. If there's a person in the loop — an agent who can edit the draft, a reviewer who can reject the extraction — the rate at which they change the machine's answer is the single most valuable number you have. It's free continuous evaluation performed by domain experts, and I'm consistently amazed how many teams collect it and never plot it. Plot it. Break it down by segment. It will tell you where the system is failing long before any aggregate metric does.

Cost per finished task. Not per request. When the model starts struggling, retries rise, escalations rise, and the bill drifts up before quality visibly drops. Cost is a quality signal in disguise.

Making the alarm real

Two things separate teams who catch this from teams who don't.

The first is that someone owns it. A named person or rota looks at the numbers weekly, and can stop a rollout. Not a dashboard that exists — dashboards that exist are wallpaper. Somebody whose job includes noticing.

The second is a small set of held-back cases that get run and graded properly on a schedule, monthly or so, by a human. A hundred examples, drawn from real recent traffic, marked by someone who knows the answer. It's a couple of hours of work and it's the only thing that gives you a true accuracy number rather than a proxy. Every team that's told me they don't have time for this has later spent considerably longer reconstructing when a regression began.

Then, when you do change something — new model version, new prompt, new retrieval — run it in shadow first. Both paths execute, only the old one is user-visible, and you compare on live traffic for a week. Practically every unpleasant surprise I've seen would have been caught by a week of shadow running, and shadow running costs money rather than reputation, which is the better currency to spend.

The uncomfortable bit

Most organisations budget for building an AI system and treat running it as overhead. It isn't. A system with a model in it needs the same standing attention as any other piece of infrastructure that touches customers, and slightly more, because its failures are polite.

The good news is that none of the above is hard. It's a handful of counters, one weekly meeting, a monthly grading session, and the discipline to shadow-run changes. The teams doing it aren't clever. They're just the ones who've already had the June conversation once and would rather not have it again.

The EU AI Act, a year in: what it actually changed for the people building

Bhaskar Paratey — Thu, 30 Apr 2026 00:00:00 +0000

The most useful sentence I can give you about the EU AI Act is this: it stopped being a legal-team topic the moment the obligations landed rather than loomed, and it became a build problem. Teams treating it as a compliance document to review at the end are the ones doing expensive rework. Teams treating it as a set of constraints on the build are the ones shipping. I've watched both happen this year, and the gap between them is money.

So let me lay out what the Act actually requires, where the legal reading and the engineering reality split, and a posture that works if you're a mid-sized company rather than a hyperscaler with a standing regulatory-affairs function.

The risk tiers, fast

Four buckets, and almost everything downstream depends on which one you're in.

Prohibited. Social scoring, certain biometric categorisation, manipulative systems. If you're here, the answer is don't.
High-risk. Systems used in defined sensitive contexts — recruitment, credit, education, critical infrastructure, certain safety components. This is the tier with teeth, and the one most teams wildly underestimate their exposure to.
Limited-risk. Chatbots, generated content. The obligation is mainly transparency: tell people they're dealing with AI or looking at synthetic media.
Minimal-risk. Everything else. No specific obligations, though general law still applies.

The trap is assuming you're limited-risk because your product feels low-stakes. The classification follows the use context, not the feeling. A mundane scoring model becomes high-risk the second it's used in hiring or lending. Map your features to contexts early, because the tier sets the cost of everything else.

What "high-risk" actually forces

This is where the legal summary and the engineering reality are furthest apart. The legal version reads "risk management, data governance, documentation, logging, human oversight, accuracy and robustness." Translated into work your engineers actually have to do:

Logging that's auditable, not just present. Records of what the system did and on what basis, retained and queryable. If your inference path doesn't emit structured, retained logs tied to inputs and model versions, that's a build item, not a config change.
Data governance you can evidence. Provenance, representativeness, bias testing of training and validation data — documented, not asserted. Most teams can describe their data. Far fewer can produce the artefact proving they examined it.
Real human oversight. Not a human who can technically click approve, but one positioned and equipped to actually intervene. A rubber stamp doesn't satisfy this, and it's obvious in an audit.
Documentation maintained as the system changes. This is the one that rots. Docs written once at launch are wrong within two release cycles.

The theme runs through all four: every one is cheap if it's in the pipeline and ruinous if you reconstruct it afterwards. You cannot retrofit a year of audit logs you never wrote.

General-purpose models and the transparency layer

If you build on general-purpose models — most of us do — obligations sit both upstream and on you. Providers of the large models carry their own documentation and transparency duties. But you don't get to outsource your end. You still disclose AI interaction and synthetic content where the limited-risk rules apply, and if your use sits in a high-risk context, the provider's compliance does not discharge yours. Get clear contractually on what the model provider hands you — model cards, evaluation data, the documentation you can actually rely on — because the gaps become your problem at audit time.

The posture for a mid-sized company

You don't have a hyperscaler's headcount. So buy nothing you can build into the pipeline, and build the cheap controls now instead of the expensive ones later.

Classify first, once, properly. A half-day mapping features to tiers saves months. Most of your surface is probably minimal or limited-risk; concentrate the effort on the genuinely high-risk slice.
Make logging and versioning a platform capability, not a per-feature scramble. If model version, inputs, outputs and the oversight decision are captured by default, compliance for new features becomes near-free. Highest-leverage thing you can do.
Treat documentation as generated, not authored. Pull model cards, eval results and data lineage from the systems that already produce them. Hand-maintained docs drift. Derived docs don't.
Put one accountable owner on it. Not a committee. Someone who owns the classification and the evidence and updates both as the product changes.

Governance is a product problem

Here's the framing I keep coming back to. People file the AI Act under legal risk, alongside GDPR or a SOC 2. Category error. The obligations — logging, oversight, data governance, documentation — are all properties of how the system is built and run. They live in the pipeline. Which makes them an engineering responsibility that legal verifies, not a legal responsibility that engineering occasionally helps with.

Bake it in and the marginal cost per feature trends towards zero. Bolt it on and you pay twice: once to build the feature, again to reconstruct the evidence it was built responsibly. The Act didn't create that choice. It just made the bill for the wrong one payable.

If AI does the junior work, where do your seniors come from?

Bhavna Ate — Mon, 20 Apr 2026 00:00:00 +0000

Here's a trap that looks like a win on a spreadsheet. AI is genuinely good at the work we used to hand to junior people — the first-draft code, the research summary, the basic analysis, the routine document. So the obvious move, the one finance loves, is to use the tool instead of hiring the junior. Same output, lower cost, no onboarding. The maths is clean.

The maths is also lying to you, because it only counts this year.

Seniors are grown, not procured

You cannot buy a senior. You can buy a senior — poach one from someone else — but as an industry, as an economy, the supply of senior people exists only because somewhere a junior person spent years doing unglamorous work badly, then less badly, then well, and became the senior. There is no other path. Expertise is the residue of having done the work, including the boring and the wrong parts of it.

The junior tasks we're so keen to automate aren't just output. They're the apprenticeship. The analyst grinding through a tedious dataset is learning what clean data feels like and developing the instinct that something's off before they can say why. The junior developer fixing trivial bugs is building a map of the codebase in their head. That "low-value" work is where judgement is manufactured. You're not paying for the output. You're paying for what doing the output does to the person.

Take it away entirely and you get a cohort who can prompt a model to produce a passable first draft but have never built the underlying judgement to know when the draft is subtly, dangerously wrong. They skipped the years that grow the instinct. And nobody notices the gap for about five years — right up until your current seniors retire and there's nobody behind them, because you stopped making any.

The pipeline problem is everyone's, and no one's

This is what economists call a collective action problem, and it's worth naming because it explains why it'll happen by default. For any single company, cutting junior hiring is rational — let everyone else bear the cost of training people, then hire the finished article. But if everybody reasons that way, nobody trains anyone, and in a decade there's a shortage of mid-level and senior talent that no salary can conjure, because the people simply don't exist. Each firm optimised itself into a problem none of them can buy their way out of.

I'm not appealing to anyone's better nature here. I'm pointing out that the firm which keeps growing its own people, while competitors strip-mine the pipeline, ends up with the one thing that becomes genuinely scarce. That's not charity. That's the smart position, held with a longer time horizon than next quarter.

Redesign the role; don't delete it

The answer isn't to protect junior work by banning the tools and having bright twenty-three-year-olds do by hand what a model does in seconds. That's just expensive nostalgia, and they'd hate it. The answer is to redesign what junior means.

Build the role around judgement instead of rote production. If the model writes the first draft, the junior's job becomes critiquing it, finding where it's wrong, improving it — which, done with a good mentor, actually develops judgement faster than producing slop from scratch ever did. They get to see more cases, across more situations, sooner. Used deliberately, AI could be the best apprenticeship accelerator we've had.

Concretely:

Protect the learning, not the task. Ask what a given junior activity was teaching. If a tool removes the activity, you have to deliberately replace the lesson — through review, through harder problems brought forward, through exposure — or you've cut the apprenticeship without noticing and kept the headline.
Make seniors teach, and count it as work. If AI absorbs the rote, senior time frees up. Aim that freed time at mentoring rather than at simply shipping more. The transfer of judgement from senior to junior is now the scarce, valuable thing, so resource it like it matters and measure people on it.
Give juniors real responsibility earlier, with a safety net. When the mechanical floor is handled, push them up the stack sooner — judgement calls, design decisions, talking to actual users — with the support to fail safely. Done right, you grow people faster than the old model did.
Hire for trajectory, not just current output. A junior who's slower than the model today but learning fast is an appreciating asset. One who leans on the tool and never builds underlying skill is a liability you won't spot for years. Hire and develop for the curve.

The organisations that win the next decade won't be the ones that cut deepest and fastest on entry-level roles. They'll be the ones who understood that AI changes what junior people should do without changing the brute fact that you still have to grow your seniors from somewhere. Automate the junior tasks and skip the junior people, and you're eating your seed corn. It tastes like a saving right up until planting season, when you find you have nothing left to plant.

Agentic AI in production: the demo lies

Bhaskar Paratey — Wed, 18 Mar 2026 00:00:00 +0000

The demo is always beautiful. Someone types a sentence, the agent thinks out loud, calls three tools, books the flight, files the expense, sends a polite Slack message, and the room claps. I've sat in that room. I've also been the one woken up six weeks later when the same agent quietly double-booked a customer's order because step four returned an empty string and nobody had decided what an empty string meant.

That gap — between the demo and the thing you can actually run for paying customers — is where most of the engineering lives, and it's where the marketing has nothing to say.

Start with the maths, because the maths is unforgiving. Say each step in your agent's chain is 95% reliable. Sounds fine. Now chain ten of them. 0.95 to the tenth power is about 0.60. Your beautiful ten-step workflow succeeds three times in five. Chain twenty steps and you're flipping a coin. Errors don't add, they compound, and an agent that plans its own steps will happily generate more steps than you expected. The single most common failure I see isn't a model "hallucinating" — it's a perfectly reasonable chain that was simply too long to survive its own length.

So the first instinct people reach for is exactly the wrong one. "Give the agent more autonomy. Let it figure it out." No. More autonomy means longer chains, more branching, more places for a 95% step to bite you, and far less ability to reason about what the thing will do before it does it. The teams shipping agents that actually hold up in production are doing the opposite. They're shortening the leash. Fewer steps. Narrower tool sets. Explicit decision points where a human, or at least a deterministic check, gets to say yes before anything irreversible happens.

That's the bit that earns its keep: guardrails, approval gates, idempotency, rollback. None of it is glamorous and none of it shows up in a launch video.

Approval gates first. Anything that spends money, sends an external message, deletes data, or touches a regulated record should pause and ask. Not because the model is stupid, but because the cost of being wrong is asymmetric. A wrong summary costs you a frown. A wrong refund costs you actual pounds and a furious customer. Gate by consequence, not by confidence — confidence scores from these models are not calibrated and you should stop pretending they are.

Idempotency is the one people forget until it hurts. Agents retry. Frameworks retry. Networks drop and the orchestrator quietly runs your step again. If "create order" runs twice you've created two orders. Every tool an agent can call needs an idempotency key or a dedupe check, the same discipline you'd apply to any distributed system, because that's what this is. An agent is a distributed system with a language model as a very confident, occasionally drunk, scheduler.

Rollback follows from that. When step seven fails, what undoes steps one through six? If the answer is "nothing, we hope it doesn't happen," you don't have a production system, you have a prototype with good PR. Saga patterns, compensating actions, transactional outboxes — the same toolkit we used for payment systems twenty years ago. The model is new. The failure modes are not.

Then there's tool use, which is where the cracks usually show first. The model calls a tool with a malformed argument. The tool returns an error the model has never seen and the model, being a pattern matcher, invents a plausible recovery that's completely wrong. Or it calls the right tool with stale data because it forgot what it retrieved four steps ago. Tools need strict schemas, validation at the boundary, and error messages written for the model to read — terse, structured, telling it exactly what to do next. Treat your tool layer like a public API being hit by an enthusiastic intern, because functionally it is.

And none of this is manageable without observability and evals, which I'll be blunt about: they are not optional, and if your team treats them as a phase-two nicety you will be debugging in production with print statements and prayer. You need to trace every step — inputs, outputs, tool calls, latencies, the lot — because when something goes wrong at step twelve of a forty-step run, "the agent messed up" is not a diagnosis. And you need evals that run a representative set of real tasks on every change, so you know whether last night's prompt tweak fixed one thing and broke nine others. Models drift, prompts rot, a provider updates something on their end and your success rate quietly drops four points. Without evals you find out from the customer.

So where do agents genuinely pull their weight today? In bounded, well-instrumented loops where the cost of error is low or fully reversible. Drafting, triage, code that a human reviews, research with citations you can check, classification, routing. Short chains, tight tools, a human in or near the loop. That's not a consolation prize — that's a large and genuinely valuable surface, and we ship it.

Where are they still a liability? Long autonomous chains touching money, irreversible external actions with no gate, anything where you can't explain after the fact why it did what it did. Putting an unsupervised agent on those is not innovation, it's a future incident with a launch date.

My honest read after building these for real customers: the technology is real and the value is real. But the engineering discipline around it lags the marketing by about two years. The vendors are selling you 2028's autonomy on 2026's reliability. Build for the reliability you actually have. Short chains, hard gates, idempotent tools, traces on everything, evals on every change. Do that and agents earn their place. Skip it and you're just automating your incidents.

The fear nobody mentions in the AI rollout meeting

Bhavna Ate — Tue, 10 Mar 2026 00:00:00 +0000

There's a question in the room at every AI kickoff I've ever sat in. It's never on the agenda. Nobody raises a hand to ask it. But it's behind every face turned politely towards the slides: is this coming for my job?

And while that question is running, nobody is listening to your rollout plan. They're doing arithmetic about their mortgage. You can present the most elegant adoption roadmap ever built and it will land on people who've already mentally checked out to worry about something you didn't name. The single biggest mistake leaders make with AI is treating it as a tooling rollout when, for the people on the receiving end, it's an existential one.

Why your reassurance bounces off

The instinct is to open with comfort. "AI won't replace you, it'll augment you." "Nobody's losing their job." "This frees you up for higher-value work."

People have heard those exact words before — about offshoring, about automation, about every reorganisation that ended with someone clearing their desk. The sentences have a track record, and the track record is mixed. So when a leader says them, the room doesn't hear reassurance. It hears the noise that historically precedes layoffs. Words spent against that history are worse than wasted; they spend down the trust you'll need later.

Reassurance is only as good as the action standing behind it. If you say nobody will lose their job and then a team's headcount shrinks two quarters later through "natural attrition" that you conspicuously don't backfill, you didn't tell the truth, and everyone now knows not to believe the next thing you say. People can handle a hard truth told straight. What corrodes a place is a soft lie told to manage them.

So tell them what's actually true, including the parts you'd rather not. Be specific about what will change — these tasks are going to be done differently, this kind of work will shrink, here is what we're asking you to learn. And be specific about what won't. Vague optimism reads as evasion. Precision, even uncomfortable precision, reads as respect.

What it costs when you get this wrong

Underestimate the human side and the bill comes due in ways that don't show up on the project plan, because none of them announce themselves.

Attrition you didn't choose. The people with the most options leave first. They're not waiting to find out how this goes — they're the ones a competitor will happily take. You're left with exactly the workforce you didn't want to over-index on.
Disengagement. Far more common than open resistance, and harder to see. People keep showing up and stop trying. They do the minimum, contribute nothing to making the new tools actually work, and let the initiative die of indifference while technically complying with it.
Sabotage, mostly passive. Not dramatic. The colleague who finds the tool "unreliable" and routes around it. The team that documents every flaw and none of the wins. A rollout the organisation is rooting against does not succeed, no matter how good the software is.

Every one of these costs more than slowing down would have. And none of them appear in a status report until they're entrenched.

A more humane way through it

I'm not arguing for endless feelings sessions. I'm arguing for a rollout that treats people as participants rather than as a population to be managed. In practice:

Bring people in before the decisions are made, not after. There is a vast difference between change done with people and change done to them, and everyone can feel which one they're in. The people doing the work know where it'll break and where it'll genuinely help. Asking them isn't a courtesy; it produces a better rollout and gives them some footing instead of pure freefall.

Be concrete about security where you genuinely can be, and straight about uncertainty where you can't. If roles are genuinely safe, say so and show how. If some roles will change substantially, say that too, and put real resources behind helping people into what's next — funded time, actual paths, not a course catalogue and good luck. The people who'll be most affected deserve the earliest, clearest conversation, not the last and vaguest.

Make managers the front line, and prepare them, because they carry this. The person who absorbs the day-to-day fear isn't the executive who announced the strategy — it's the team lead getting the worried questions in the one-to-one. If that manager is anxious and unsupported, they pass the fear straight down, amplified. Look after the managers and a lot of this becomes manageable.

Then watch the right signals — engagement, the questions people actually ask, where they go silent — and adjust. A rollout you can't course-correct isn't a plan, it's a gamble.

The fear is rational. People watching a capable machine learn their job are responding sensibly to real uncertainty, and pretending otherwise insults them. You won't make it disappear with a confident slide, and you shouldn't try. You earn your way through it — by being straight about what's happening, by acting in line with what you said, and by treating the people going through the change as the ones most worth listening to about how it should go. Skip that and you don't avoid the cost. You just pay it later, with interest, in the quality of the people who decide to stay.

What we actually spend on AI, and where it leaks

Bhaskar Paratey — Sat, 14 Feb 2026 00:00:00 +0000

A team once showed me a per-request cost they were proud of — a fraction of a cent. I asked what fraction of those requests succeeded first time. They didn't know. We pulled the numbers: a chunk failed, retried, escalated to a bigger model, then landed on a human reviewer. Their real cost per finished task was many multiples of the number on the slide. They'd been optimising the cheapest thing in the system.

That's the whole story of LLM cost. The token price on the pricing page is easy to see and the least worth obsessing over. The bill that surprises people is assembled from everything around the token: the retries, the evals, the human review, the context you stuffed in because it was simpler than building retrieval. Treat LLM spend like any other unit-economics problem and most of these stop being surprises.

Cost per request is the wrong metric

It's trivial to compute and it tells you almost nothing. It optimises the cheap part and ignores the expensive parts. The metric that matters is cost per resolved task — the fully loaded cost of getting one unit of actual work done, successfully, to a standard you'd accept.

This isn't academic. A request costing a fraction of a cent that fails a quarter of the time, triggers a retry, escalates to a larger model, then lands on a human has a resolved-task cost many times the headline number. Optimise per-request and you'll happily shave the cheap input while the real cost sits in the failure path you aren't measuring.

Where the spend actually leaks

Roughly in order of how often I see teams underestimate it:

Retries and fallbacks. Every failed call you retry, every escalation to a bigger model when the small one fell short, is real spend that never shows in your per-call estimate. A double-digit retry rate doubles your cost without ever announcing itself.
Context stuffing. When retrieval is hard, teams paste the whole document in. It works. It's also expensive on every single call, forever. The most common leak I find, because it's invisible until you look at average input token counts.
Prompt bloat. System prompts that grow by accretion — every incident adds a clause, nothing's ever removed. You pay for those tokens on every request, multiplied by volume.
Evaluation. Running evals is non-negotiable for quality, but eval traffic is real model traffic. Teams budget production and forget that a serious eval suite can be a meaningful fraction of the production bill.
Human review. The most expensive token in the system is a minute of a person's time. If your design routes a real share of outputs to human review, that — not the API — is your dominant cost, and it scales with volume the worst.

None of these show up if you only watch the pricing page. All of them show up in cost per resolved task.

Two levers that actually move the number

Caching. A surprising fraction of production traffic is repetitive or near-repetitive. Caching exact and semantically similar responses, plus prompt caching for stable context, takes load off the model directly. Cheapest win available, and the first thing to instrument.

Model routing. Send the cheap, small model first; escalate to the big one only when the task demands it. Most workloads have a long tail of easy cases that don't need your most capable model. The discipline is measuring the escalation rate and the quality at each tier — so routing cuts cost without silently degrading the resolved-task rate. Done blind, routing just relocates the leak.

Self-hosting: when it wins and when it doesn't

The instinct, once the bill grows, is to self-host an open-weight model and escape API pricing. Sometimes that's right. Often it isn't, and the deciding factor is utilisation.

Self-hosting swaps a variable per-token cost for a largely fixed one: GPUs, the engineering to run them, the on-call to keep them up. That maths works when you have high, steady volume keeping expensive hardware busy. It works badly when your traffic is spiky or modest — you pay for idle GPUs around the clock to serve a few busy hours, and you've taken on operational burden the API was absorbing for you.

So the straight version. Self-hosting beats API pricing when you have sustained high utilisation, a privacy or latency requirement the APIs can't meet, and the team to operate inference properly. It loses when volume is low or bursty, when you'd run below capacity, or when "we save on tokens" conveniently ignores the salaries of the people now keeping the cluster alive.

The order of operations

When the AI bill needs managing, work it in this order:

Instrument cost per resolved task, broken down by feature. You can't manage what you haven't attributed.
Find the dominant term. Tokens, retries, eval, or human review — usually not what people assume.
Attack that term specifically. Trim prompts and context if it's input tokens. Fix quality to cut retries if it's the failure path. Reduce review load if it's people.
Add caching and routing as standing infrastructure, escalation rate monitored.
Only then model self-hosting, and only if utilisation justifies it.

AI spend is unit economics, not a technology curiosity. Define the unit, attribute the cost, find the dominant term, fix it. The teams that stay solvent here aren't the ones who negotiated the best token price. They're the ones who know their cost per resolved task to the cent and can tell you exactly which term is the leak.

Reskilling isn't a memo

Bhavna Ate — Fri, 30 Jan 2026 00:00:00 +0000

A leadership team announces that the organisation will "reskill the workforce for AI." A budget line appears. A platform gets bought, the one with ten thousand courses nobody asked for. Everyone is enrolled. A dashboard somewhere lights up green when completion rates climb. Six months on, the work is being done exactly as it was before, except now there's a slide that says the workforce has been reskilled.

I've watched this play out enough times to recognise the shape of it before it finishes. The tell is that the programme measures whether people watched things, not whether anything changed. And reskilling, the real kind, is not a content-delivery problem. It's a change-of-behaviour problem, and behaviour is the most expensive thing in any organisation to shift.

Training is not reskilling

Let me draw the line clearly, because the whole confusion lives here.

Training is when someone learns what a tool does. Reskilling is when someone's actual job changes and they're capable of doing the new version of it. The first is a webinar. The second is months of doing real work differently, with support, while still being expected to deliver.

A two-hour session on prompting an LLM is training. It is genuinely worth doing and it changes almost nothing on its own, because the moment people return to a workflow, a backlog, and a manager measuring them on the old output, they revert. The pull of how-we've-always-done-it is enormous, and a certificate does not counter it. What counters it is changing the work itself — the templates, the definition of done, what gets reviewed, what gets rewarded — so that using the new capability is the path of least resistance rather than an extra thing to remember on top of a full day.

If your programme didn't touch the actual workflow, you ran a training programme and called it reskilling. They are not the same line item, and they don't cost the same.

The realities nobody budgets for

Three things get systematically underestimated. I'll be specific.

Time. Reskilling is not free time appearing from nowhere; it's productive hours spent learning instead of delivering, which means output dips before it recovers. A team genuinely changing how it works will be slower for a stretch. If you haven't planned for the dip and protected people through it, you've designed a programme that punishes the very people taking part — they fall behind on their normal targets while doing the thing you asked.
Managers. This is the one that decides everything, and it's the one most programmes ignore. If a manager doesn't change what they ask for and how they assess it, nothing downstream of them changes, regardless of how many courses their reports complete. The manager is the gatekeeper of whether the new skill is allowed to be used. Reskill them first, or accept that you're shouting past the person who controls the day.
The unglamorous middle. Not everyone is a curious early adopter who'll teach themselves over a weekend. Most of an organisation is competent, busy people who'll engage if it's made genuinely doable and ignore it if it's piled on top of everything else. Design for them, not for the enthusiasts who didn't need you anyway.

Who gets left behind, and the deliberate work of not doing that

Here's the uncomfortable part. "Reskilling" is often the polite word organisations use while a cohort of people quietly gets left behind — usually the ones who've been doing a now-automatable task for fifteen years, are excellent at it, are in their fifties, and have absorbed an unspoken message that the future doesn't include them. The programme technically applies to them. The design does not consider them.

Leaving them behind is rarely a decision anyone makes out loud. It's what happens by default when you don't decide otherwise. So you have to decide otherwise, on purpose. That means looking at who is most exposed before you start, building paths that connect their deep domain knowledge to where the work is going rather than treating them as beginners, and being plain with people about what's changing instead of letting them infer their fate from a vague all-staff email. The domain knowledge in a long-tenured person is often the scarcest asset in the building. Burning it because the retraining was designed for twenty-five-year-olds is an expensive mistake dressed up as progress.

Did it work?

Completion rates tell you nothing. Here's what I'd actually look at, three to six months after the courses end:

Has the work changed? Pull real artefacts — documents, tickets, code, decks — and check whether they look different from before. If they're identical, the skill isn't being used, whatever the dashboard says.
Are people using the capability without being told to? Voluntary, unprompted use is the only honest signal that something stuck.
Did anyone get left behind, measurably? Look at engagement and attrition in the most-exposed groups specifically. A programme that "succeeded" overall while a cohort silently disengaged did not succeed.

Reskilling done properly is slow, costs real money beyond the platform licence, makes the team temporarily slower, and demands that managers change first. None of that fits neatly on a slide. That's precisely why so few organisations actually do it, and why the ones that do will be the ones still standing with their people intact on the other side.

Small models, big deal

Bhaskar Paratey — Thu, 22 Jan 2026 00:00:00 +0000

Every few weeks another frontier model lands, another trillion parameters, another leaderboard, another round of breathless coverage. Fine. Some of it matters. But the most useful AI work my team has shipped in the last year didn't touch a frontier model at all. It ran on an 8B model, fine-tuned on one narrow task, quantised down to run on hardware a customer already owned. No data left the building. Latency under a hundred milliseconds. Cost per query rounding to zero.

Nobody wrote a press release about it. It just worked, and it kept working when the internet didn't.

Here's the bit the size race keeps missing. For an enormous number of real tasks, you don't need a model that can write a sonnet, debug Rust, and explain Ottoman tax policy. You need a model that does one thing — classify this ticket, extract these fields, redact this document, answer questions about this manual — reliably, cheaply, and fast. A general model the size of a small country is overkill for that, and overkill has costs that don't show up on the leaderboard.

The wins from going small are not marginal. They're structural.

Privacy first, because it's the one that closes deals. When the model runs on the device or on a server inside the customer's own walls, the sensitive data never crosses the network. For anyone in healthcare, finance, defence, or any regulated space, "the data never leaves" isn't a feature, it's the entire precondition for the conversation. I've watched procurement processes that would've taken nine months collapse to weeks the moment we could say the inference happens on-premise. You cannot leak what you never send.

Latency next. A round trip to a cloud API is a hundred, two hundred milliseconds before the model has thought about anything. On-device, you skip the journey entirely. For anything interactive — a keyboard, a camera, a control loop, a thing a human is waiting on — that difference is the difference between magic and irritating. And it doesn't fall over when the connection does. Edge devices in a factory, a vehicle, a remote site, a hospital basement with terrible signal — they need to keep working regardless. A model that requires a healthy uplink to function is a model that fails exactly when you need it most.

Then cost. Frontier inference billed per token is fine until you're doing it a few million times a day, at which point the bill becomes a strategic problem. A small model on hardware you already paid for has a marginal cost that approaches nothing. At volume that's not a saving, it's a different business model.

Now, the honest part, because going small is not free and anyone who tells you otherwise is selling something. The constraints are real and you have to engineer around them.

Quantisation is the main lever and the main trap. Dropping from 16-bit to 8-bit weights usually costs you almost nothing measurable and roughly halves your memory. Push to 4-bit and you'll often still be fine — but "often" is doing real work in that sentence. On some tasks 4-bit quietly degrades in ways that don't show up until you look closely, and the degradation isn't uniform across what the model can do. You don't get to assume. You measure, on your task, with your data.

Memory is the hard wall. An 8B model in 4-bit is roughly five gigabytes before you've loaded a single token of context, and context eats memory too. On a phone, a browser tab, an embedded board, that ceiling decides everything. Half of small-model engineering is just making the thing fit and stay fit while it runs.

Which is why eval discipline matters more here, not less. With a giant general model you can lean on its slack. A small fine-tuned model has no slack — it's sharp on the task you trained it for and it falls off a cliff just outside that. So you need a real evaluation set drawn from real inputs, and you need to run it every time you change the model, the quantisation, or the fine-tune. The failure mode I see constantly is a team that fine-tuned once, eyeballed a few outputs, declared victory, and never built the harness that would've told them when it broke.

The payoff, when an 8B model fine-tuned on your task beats a frontier model, is real and more common than people expect. On a narrow, well-defined job with good training data, the small specialist regularly outperforms the giant generalist — and it's faster, cheaper, and private while doing it. The generalist knows a little about everything. The specialist knows your thing cold.

So here's the heuristic I actually use when someone asks what size model to reach for. If the task is narrow, repetitive, and you can describe "right" precisely — start small, fine-tune, and only scale up if the evals demand it. If the task is open-ended, varied, needs broad world knowledge or genuine multi-step reasoning across domains you can't enumerate — reach for the big general model and pay for it. And if it sits in between, prototype on the big model to prove the task is even solvable, then distil down to the smallest thing that holds the bar. Prove it works, then make it small.

The frontier will keep climbing and that's fine. But most of the value in the next few years won't be at the frontier. It'll be in thousands of small, sharp, boring models running quietly close to the data. That's where the engineering is interesting and the economics actually close.

The exit you have never tested

Bhaskar Paratey — Thu, 11 Dec 2025 00:00:00 +0000

The open-weights argument is nearly always conducted as a values debate. Openness good, gatekeepers bad, or alternatively, safety serious, amateurs reckless. I find I have very little to contribute to that, partly because both sides are arguing about the world and I'm usually being paid to argue about one company's next eighteen months.

The question I actually care about is duller and more answerable. If the model you're building on became unavailable, unaffordable, or unacceptable in six months, what would it cost you to move?

Most teams have never worked this out. Which means they've taken on a risk of unknown size, which is the only kind that really hurts.

Why the question comes up now

Three things happen, and they've all happened to clients of mine in the past year.

A hosted model gets deprecated. You're given a migration window, which sounds generous until you realise your prompts were tuned against the old version's particular quirks and the replacement is better on benchmarks and worse on your task. That's not hypothetical; better-on-average routinely means worse-on-yours, and you find out with three weeks left.

The price changes, or your volume does. At pilot scale nobody cares about per-token cost. At production scale the bill becomes a line item somebody senior has opinions about, and suddenly the question of whether this workload could run somewhere cheaper is not academic.

Or the requirement changes underneath you. A customer's procurement asks where the data goes. A regulator asks the same. A deal you want requires processing to stay in a particular jurisdiction, and your entire architecture assumes an API endpoint you don't control.

None of these are exotic. They're the ordinary weather of running software with a supplier in the critical path, and we all knew how to think about this before AI turned up — we just seem to have forgotten while we were excited.

What actually locks you in

Here's the useful part, because the lock-in isn't where people look for it.

It isn't the API. Swapping one chat completion endpoint for another is an afternoon, and there are half a dozen libraries that will do it for you. If your migration plan is "we use an abstraction layer," you have solved the five percent of the problem that was never going to be the problem.

The lock-in is in three other places.

Your prompts. They are not portable and pretending otherwise wastes a fortnight. A prompt that's been refined over months against one model encodes hundreds of small accommodations to that model's tendencies — how it handles a negative instruction, whether it needs the format example, how it drifts when the context gets long. Move it and it degrades in ways that are individually minor and collectively fatal. Expect to redo real work per prompt, and budget for it honestly.

Your fine-tunes. If you've fine-tuned a hosted model, you have a derivative of an asset you don't hold. The training data is yours, the artefact isn't. This is fine if you kept the data, the pipeline, and the ability to run it again. Many teams did not keep the pipeline, because it was run once, by a contractor, in a notebook.

Everything you've built around one model's failure modes. The guardrails, the retry logic, the output validators, the little cleanup function that strips the preamble it always adds. That accumulated scar tissue is quietly the most expensive thing to reproduce, and it's invisible in any architecture diagram.

The thing that makes an exit possible

There's exactly one investment that converts a scary migration into a boring one, and it's the same artefact I bang on about constantly: a real evaluation set.

If you have a few hundred graded examples that represent what your system is for, then swapping models is a Tuesday. You point the harness at the candidate, you get numbers, you see precisely where it's worse, you fix those cases or you decide the trade is acceptable. The decision becomes evidence-based and takes days.

Without it, the migration is a matter of opinion, conducted under time pressure, by people who are frightened. I've watched that go badly enough times to be blunt about it: your evals are your portability. Everything else is architecture theatre.

So, open or closed?

Given all that, my actual advice is unromantic.

Use the best hosted model for the work that's hard, novel, and low-volume, where capability matters more than anything and the frontier is genuinely ahead. Pay for it. Don't self-host to prove a point; running inference well is a real operational discipline and most teams underestimate it by a factor of several.

Move the high-volume, narrow, well-specified work onto weights you hold, once it's stable — not for ideology, but because at volume the economics and the latency and the "your data never leaves" conversation all point the same way, and because a model you host doesn't get deprecated on someone else's schedule.

And regardless of which you pick: keep your training data and your pipeline, keep your evals current, and once — just once — actually run the migration. Take a real workload, point it at a different model, and see what breaks. A day spent finding out is cheaper than a quarter spent discovering it during an incident.

The teams that will handle the next shift calmly aren't the ones who picked correctly. They're the ones who know what picking wrongly would cost.

RAG done right: it's a search problem, not an AI problem

Bhaskar Paratey — Mon, 10 Nov 2025 00:00:00 +0000

I've now reviewed enough disappointing retrieval-augmented generation projects to spot the diagnosis before anyone opens a laptop. The team is brilliant. The model is the latest and best. The vector database is the fashionable one. And the answers are still wrong, vague, or confidently citing the wrong document. Then someone suggests trying a bigger model, and that's the moment I know exactly what went wrong.

They built an AI project. They needed a search project.

RAG is, mechanically, very simple. You find the relevant bits of your knowledge base, you stuff them into the prompt, the model answers using them. That's it. And in that pipeline the model is almost never the bottleneck. The bottleneck is the "find the relevant bits" step — the retrieval — which is a search and information-retrieval problem that the field spent thirty years working on before transformers were a twinkle in anyone's eye. The teams that nail RAG are the ones who remember that. The teams that struggle are the ones who think the embedding model absolves them of doing search properly.

If the right chunk never makes it into the prompt, the cleverest model on earth will answer from thin air. Garbage retrieval in, confident nonsense out — and now it's nonsense with a citation, which is worse, because it looks trustworthy.

So let's talk about the unglamorous things that actually decide whether your RAG works.

Chunking, first, because it's the most underrated decision in the whole stack. How you split your documents determines what can ever be retrieved. Split too small and you shred the context — half a sentence retrieves cleanly and means nothing. Split too big and each chunk is a muddle of five topics, so the embedding represents none of them well and the model drowns in noise. There's no universal right answer; it depends on your documents. But splitting on real structure — sections, headings, semantic boundaries — beats blindly cutting every 500 tokens almost every time. I've seen a RAG system go from useless to genuinely good with no model change at all, purely by fixing how documents were chopped up.

Metadata, next, and this is where most teams leave the biggest win on the floor. Your chunks have properties — source, date, author, document type, department, version. If you're not capturing and filtering on those, you're asking a similarity search to do a librarian's job. "What's our current refund policy" should never surface the 2019 version, and no amount of semantic similarity reliably prevents that. A date filter does, instantly. Half the retrieval failures I see are really metadata failures wearing a trench coat.

Now the one people argue with me about: hybrid beats pure vector far more often than the vector-database marketing admits. Pure semantic search is genuinely good at meaning and genuinely bad at exact matches — product codes, error numbers, proper nouns, acronyms, that one specific term of art your industry uses. Someone searches for error code "E-4471" and a vector search helpfully returns chunks about errors in general. Old-fashioned keyword search — BM25, the technology that ran search engines for decades — nails the exact match every time. Run both, combine the rankings, and you get the meaning and the precision. It's more engineering than flipping on a vector store, which is precisely why people skip it and then wonder why exact-match queries fail.

Which brings me to the discipline that separates the projects that improve from the ones that plateau: evaluate retrieval separately from generation. These are two different systems and they fail differently. If you only judge the final answer, you can never tell whether a bad answer came from bad retrieval or bad generation, and so you can't fix it — you just keep swapping models and changing nothing that matters. Build a set of real questions with the documents that should answer them. Measure whether retrieval surfaces the right chunks — recall, precision, where the right chunk ranks. Get that solid first. Only then worry about how the model phrases things. Fix the search before you touch the prose.

And under all of it sits the work nobody wants: data hygiene. Duplicate documents, three contradictory versions of the same policy, scanned PDFs that OCR'd into gibberish, tables that flattened into word soup, stale content nobody flagged as dead. RAG over a messy knowledge base produces messy answers with total confidence. The least glamorous and most valuable thing you can do for a RAG project is clean the underlying data — deduplicate, version, fix the extraction, prune the dead weight. It's tedious and it's most of the actual job, and the teams that skip it are the teams that ship something embarrassing.

Last thing, and I mean it: sometimes you don't need RAG at all, and reaching for it is just complexity cosplay. If the relevant information fits comfortably in the context window, put it in the prompt — context is cheap now and a retrieval pipeline you don't need is a liability you have to maintain. If the knowledge is stable and bounded, fine-tuning may serve better than retrieving the same things forever. If the answer lives in a database or an API, give the model a tool call and let it fetch the precise answer rather than fuzzy-matching against embedded text. RAG is the right tool for a large, changing corpus of unstructured text you need to ground answers in. It is not a default, and treating it as one is how you end up maintaining a vector database to solve a problem a WHERE clause would've handled.

Get the search right and the AI part mostly takes care of itself. Get the search wrong and no model will save you. That's the whole lesson, and it predates the hype by about three decades.

In hiring, the bias you automate is the bias you scale

Bhavna Ate — Mon, 15 Sep 2025 00:00:00 +0000

A recruiter rejecting one CV an hour on a hunch is a small, containable problem. You can coach that person. You can audit their decisions. You can sit them down and ask why every name they shortlisted sounds like their own. A model rejecting ten thousand CVs an hour on a pattern nobody can fully articulate is a different kind of problem entirely, and most of the companies buying these tools haven't worked out that it's worse.

That's the thing I keep saying in rooms where someone has just demoed a shiny "AI-powered talent platform" and the heads are nodding. Automating a process doesn't sanitise it. If the process was biased, you've now made the bias faster, cheaper, and far harder to see. You've taken something a human did clumsily and given it the gloss of mathematical objectivity, which is exactly the disguise discrimination wants.

Where the danger actually sits

I spent two decades inside large technology organisations before I moved into People work, and the engineering habit that stuck with me is this: be specific about which part of the system you're worried about. "AI in recruitment" is too broad to have an opinion on. So let me split it.

Ranking and scoring candidates is the dangerous end. A model trained on who your company hired and promoted in the past learns, with great fidelity, who your company hired and promoted in the past. If that history skewed male, or skewed towards one set of universities, or invisibly penalised a two-year gap that usually means childcare, the model absorbs all of it and presents the result as a score out of a hundred. The most cited example is still Amazon's experimental tool that taught itself to downgrade CVs containing the word "women's", but I don't lean on it because it's old — I lean on it because it's typical. The model did exactly what it was built to do. The training data was the discrimination.

Video "personality" scoring is worse, and I'll be blunt about it: I think most of it is closer to phrenology than science. Tools that infer conscientiousness or "culture fit" from facial micro-expressions, vocal tone, or word choice are making confident claims about interior states they cannot observe. They penalise accents, neurodivergence, a bad webcam, a candidate interviewing from a noisy flat because they share it with three other people. Illinois and a handful of other places now regulate exactly this for good reason. If you are scoring a human being's worth from how their face moves on a laptop camera, you have lost the plot.

Where it genuinely earns its place

I'm not against the technology. I'm against pointing it at the decision. Used on the work around the decision, it's straightforwardly useful.

Scheduling. Coordinating five interviewers, two time zones and a candidate's day job is miserable admin, and there is no fairness question buried in a calendar invite. Automate all of it.
Sourcing breadth. Search tools that surface candidates you'd never have found — different regions, non-obvious title histories, people who don't use the keywords your team happens to use — widen the top of the funnel. That's the opposite of bias, as long as the widening is where it stops and a person decides who to actually approach.
Drafting and tidying. First-pass job descriptions, summarising a long CV so a human reads it faster, flagging that a posting is full of jargon that deters the very people you want. Assistive, reversible, low-stakes.

The pattern, if you want one line to carry out of here: let the machine widen the funnel and handle the logistics. Don't let it narrow the field. Breadth is safe. Judgement is not.

Somebody's name has to be on it

Here's the part people skip because it's inconvenient. For every consequential decision, a named human being has to be accountable — not the vendor, not "the algorithm", a person you could put in front of a tribunal. The moment a rejected candidate, or a regulator, or your own board asks "why was this person screened out", the answer cannot be a shrug at a black box. Under the EU AI Act, recruitment systems are high-risk and that accountability is becoming a legal requirement, not a nicety. But I'd want it even if no law demanded it, because a decision nobody will own is a decision nobody can defend.

So when I deploy any of this, the rules are dull and non-negotiable. The tool recommends; a person decides and signs. We keep records of why, in language a human wrote. We test outcomes across groups before go-live and on a schedule after — not because we expect a clean bill of health, but because the only honest assumption is that bias is present until measured otherwise. We tell candidates an automated step exists and give them a route to a human. And we keep one uncomfortable question pinned to the wall: if this model is wrong about someone, who finds out, and how?

Most vendors can't answer that last one. That's usually all I need to know.

The mess of it is that good hiring was always hard, slow, and human, and AI is being sold as the thing that finally makes it fast, cheap, and clean. It can make it faster and cheaper. Clean is the lie. If you remember nothing else: a tool that scales your hiring also scales whatever was already wrong with it, and "the system did it" has never once been an acceptable reason to end someone's chance at a job.

The voice she recorded while she still could

Dimple Paratey — Tue, 10 Jun 2025 00:00:00 +0000

A friend of a friend — I'll call her Jo, because I've not asked her permission to write this and I'd rather be careful — was diagnosed with motor neurone disease three years ago. One of the things a speech therapist told her early on, at a point when she could still speak more or less normally, was to bank her voice. Sit down and record a few hundred phrases so that when the disease takes her speech, the device she'll use to talk sounds like her.

Ten years ago that meant hours of recording in a clinical setting and a synthetic voice that was recognisably hers in the way a photocopy of a photograph is recognisably you. Now it takes a fraction of the recording and the result is close enough that her son, on the phone, hears his mum.

I've thought about that a lot. Not because it's uplifting — it isn't, particularly, it's a terrible illness and a good tool doesn't change that — but because it's the clearest example I know of AI doing something that could not be done any other way, for people who were not the target market of anything.

The unglamorous version, which is most of it

Voice banking is the story that makes people put their glass down at a party. The everyday version matters more, because it affects far more people far more often.

Live captions that are actually accurate, on any call, without arranging anything in advance. Image descriptions generated on the fly so that a blind person browsing a website isn't confronted with forty instances of the word "image." Speech recognition that copes with dysarthria — with a voice that slurs, or shakes, or stops — which mainstream systems were terrible at for years because they were trained on people who sound like newsreaders. Text simplified into plain language for someone with a cognitive disability or a brain injury, on demand, without having to ask a person and feel like a burden.

Every one of those is a thing that used to require either money, advance planning, or asking another human for help. Removing the "asking another human" step is underrated by people who've never had to do it eight times a day.

There's an old idea in design about dropped kerbs — cut into pavements for wheelchair users, and then used by everyone with a pram, a suitcase, a delivery trolley. Most of this is a dropped kerb. Live captions are used by disabled people out of necessity and by everyone else on a train with no headphones. The features arrive because of a small group and stay because of a large one, which is usually how the funding gets justified, and I've made my peace with that.

Where I get uneasy

Here's the part I'd want anyone building this to hear, because it's the failure mode I keep seeing and it's a subtle one.

Automatic accessibility can become an excuse to stop doing accessibility.

A team turns on AI-generated alt text across their site. The audit score goes up. Somebody screenshots the dashboard. And the alt text now says "a person sitting at a desk" for an image whose entire point was that the chart on the screen shows a 40% drop. Technically described. Functionally useless. Before the tool existed, someone might have written a sentence that mattered. Now nobody will, because the box is ticked and the box is what gets measured.

Same with captions. Automatic captions are genuinely good now, in good conditions, in a common accent, on ordinary vocabulary. In a lecture on pharmacology, with a lecturer from Newcastle and a room with an echo, they are not good, and the errors are confident and plausible, which is worse than errors that look like errors. A deaf student who can't tell which words were wrong is in a worse position than one who knows the transcript is unreliable.

So the test I'd apply is simple, and it's not a technical test. Does the disabled person get to check it, correct it, and be believed? A system that generates a description and lets the user say "that's wrong, describe it again, tell me what's on the screen" is a tool. A system that generates a description, publishes it, and offers no recourse is a compliance product that happens to sit between a person and the thing they were trying to reach.

And the obvious thing, which somehow needs saying

Ask them.

The number of accessibility features I've seen designed by people who have never watched a screen reader user navigate their product is not small. It's most of them. An afternoon with four people who actually use this stuff will tell you more than a quarter of internal workshops, and it will spare you shipping something well-meant and infuriating.

Jo's voice works, by the way. She says the synthesised version is a bit too polite — that it doesn't do her swearing properly, which she considers a significant regression. I think that's the most useful piece of product feedback I've heard all year.

The only question I ask in an AI demo

Bhaskar Paratey — Wed, 02 Apr 2025 00:00:00 +0000

I've sat through a lot of demos in the past two years and I've narrowed my contribution down to one question, usually asked about four minutes in.

"How do you know it works?"

Roughly half the time this ends the demo. Not because the answer is bad — because there isn't one. The team has been iterating for months on a system whose quality they assess by trying it, looking at the output, and going "yeah, that's better." Vibes. Very sophisticated vibes, held by clever people with good instincts, but vibes, and vibes don't survive a version bump.

Here's why it matters more than it used to. With conventional software, a regression announces itself. Something throws, a test goes red, a page 500s. With a language model in the loop, a regression is a slightly worse answer, and slightly worse answers look exactly like answers. You can degrade for six weeks and only find out when a customer complains about something that had been quietly broken since the prompt tweak on the 3rd.

Start with the failures you already have

The instinct is to build a big representative test set, and that instinct will keep you busy for a month and produce something bland. Don't start there.

Start with a folder. Every time the system does something wrong — a bad answer, a hallucinated figure, a refusal that shouldn't have happened, an escalation that should have been handled — the person who spotted it drops the input, the output, and one line about why it's wrong into that folder. That's it. No process, no ticket type, no ceremony, because ceremony is how this dies.

Within a fortnight you'll have thirty real failures. Those thirty are worth more than three hundred synthetic examples, because they're the actual distribution of ways your users break your system, which is never the distribution you'd have imagined. Everything grows from there.

Somewhere around a hundred and fifty examples you have a genuine evaluation set: mostly real failures, a decent number of cases it handles correctly (so you can catch the fix that breaks something else), and a handful of deliberately nasty edge cases. That set is now the most valuable artefact in the project. More valuable than the prompt, which you will rewrite forty times.

Get the ground truth argument out of the way

For each case, someone has to say what the right answer would have been. This is where projects discover that they don't agree.

Two of your experts will look at the same output and one will call it acceptable and the other won't. That disagreement is not an annoyance to be smoothed over; it's the specification of your product surfacing for the first time. Have the argument. Write down the resolution. If you can't resolve it, you've found a place where your product doesn't know what it's for, and no amount of model work will paper over that.

Where the output is open-ended, don't try to grade it out of ten. Nobody is consistent at that, including the same person on a different afternoon. Ask narrower binary questions: did it use only the supplied sources? Did it get the number right? Did it refuse when it should have? Four crisp yes/no checks tell you far more than one holistic score, and they tell you what broke rather than just that something did.

On using a model to grade a model

You'll want to automate this, and you can, within limits. Using a strong model as a judge works decently for constrained questions of the kind above. It works poorly for "is this good," which is unsurprising.

But calibrate it before you trust it. Take fifty cases you've graded by hand, run the judge over them, and see how often it agrees with you. If it's agreeing 90% of the time on your judgements, you have a useful instrument. If it's at 70%, you have a random number generator with a nice interface, and any improvement it reports is inside its own noise. I've seen teams celebrate a four-point gain from a judge whose agreement rate meant four points was indistinguishable from nothing.

Also, and this catches people: don't have the same model family grade its own output on style. It likes its own writing. So would you.

Make it a gate, not a report

The last step is the one that changes behaviour. The eval has to run automatically on every change to the prompt, the retrieval, the model version, the temperature, the chunking — every knob — and it has to block the change if the numbers drop. Not email a dashboard. Block.

The moment that's in place, a whole class of argument disappears from the team. "Does this prompt change help?" stops being a matter of opinion and seniority and becomes a number that took eight minutes to produce. That shift is worth more than the eval itself, honestly. It's the difference between a team that improves and a team that oscillates.

None of this is exotic. It's the testing discipline that ordinary software worked out decades ago, applied to a component that fails softly instead of loudly. The reason it gets skipped is that it isn't fun, it produces no demo, and it makes visible how often the system is wrong — which is uncomfortable in month two and priceless in month nine.

If you can't answer the question, you don't have a product yet. You have a very persuasive prototype, and those are much easier to build than they used to be.

The meeting has a memory now

Bhavna Ate — Thu, 13 Feb 2025 00:00:00 +0000

Somewhere in the last eighteen months, without a decision being taken anywhere, the meeting acquired a permanent memory.

Nobody ran a consultation. There was no policy. Someone connected a notetaker to their calendar because it was useful, a colleague saw the summary and asked what it was, and within a few months it was in every recurring call in the organisation, sitting quietly in the participant list. I've watched this happen at four or five clients now and the sequence is almost identical each time.

I want to be fair to it, because the benefits are not imaginary. And then I want to talk about the part that people feel before they can articulate it.

The genuinely good part

For a lot of people this technology is the best thing to happen to their working week in years.

If you have a hearing impairment, a live transcript is not a convenience, it's access. If English is your third language and the call is six British people talking over each other, being able to read it back at your own pace changes what you can contribute. If you have ADHD and the effort of taking notes competes directly with the effort of listening, being relieved of one of those is enormous. If you're on parental leave or in a different timezone or simply double-booked, catching up in four minutes rather than being permanently half-informed is the difference between being in the loop and drifting out of it.

I don't think any of that gets said enough, because the discourse about meeting notetakers is mostly written by people for whom meetings were already working fine.

There's also the ordinary version: actions get captured, the "what did we agree?" argument three weeks later gets settled in ten seconds, and nobody has to nominate the most junior person in the room to type while everyone else thinks.

What changes in the room

Here's the part I'd want any leadership team to sit with.

When people know the room has a permanent, searchable, exportable memory, they speak differently. Not dramatically. Nobody announces that they've become more careful. But the half-formed idea doesn't get floated. The honest "I don't actually understand this" gets swallowed. The bit where someone says "look, between us, the timeline is fantasy" — that stops, and it stops first among the people with the least security, which is precisely the people you most need to hear from.

I asked a group at one client, anonymously, whether the notetaker had changed how they talked in meetings. Just over half said yes. Of those, almost all described the change as being more guarded. Two people said it made them more likely to speak up because they knew they'd be credited properly, which I thought was an interesting and genuine counterweight, but the balance was clear.

Psychological safety is not a soft concept. It is the mechanism by which problems reach you early enough to be cheap. If a tool trades some of that for tidier action items, that's a trade you should make consciously, and most organisations have made it accidentally.

The summary is not the meeting

Second thing, more mundane and probably more immediately damaging: these summaries are confidently wrong in specific, recurring ways, and people treat them as the record.

They flatten disagreement. A discussion where three people had reservations and one senior person pushed through becomes "the team agreed to proceed." They mis-attribute. They struggle with the person who said the crucial thing quietly at minute fifty-two after everyone had mentally moved on. They cannot hear irony, and British professional life runs substantially on irony.

None of that would matter if the summary were treated as a convenience. It matters because within about a month the summary becomes the thing people cite, and the meeting itself becomes something nobody remembers independently. You've created an authoritative record with a systematic bias toward consensus and seniority, and no named human is accountable for its accuracy.

What I'd actually put in place

I'd resist the instinct to ban it. Bans on useful things produce shadow use, and you lose the access benefits along with the risks.

Four things, and they fit on one page.

Say clearly which meetings are recorded by default and which are not. One-to-ones, anything about performance, anything about someone's health or circumstances, and any conversation whose purpose is to surface bad news — these should be off unless both parties actively want it on. Make "off" socially easy: if declining requires a person to make a small speech, you have not given them a choice.

Name an owner for every summary. A person, not a tool. They read it before it circulates and they fix what's wrong. This takes two minutes and it's the difference between a record and a rumour.

Be explicit about retention and access. Who can search these? For how long? Can they be pulled into a grievance, a performance case, a redundancy consultation? People will assume the worst answer if you don't give them the real one, and quite often the worst answer is the true one, in which case they deserve to know.

And once a quarter, ask people whether it's changed how they speak. Anonymously. Then actually look at the answer.

The technology here is not the hard bit. The hard bit is that we installed a memory into the one place where organisations do their honest thinking, and we did it without deciding to.

The data was always the project

Bhaskar Paratey — Tue, 19 Nov 2024 00:00:00 +0000

I get called in when things have stalled. It's a decent chunk of what I do, and after enough of these I've stopped bothering with a diagnostic phase, because the answer is nearly always the same and I can usually confirm it in an afternoon.

The model is fine. The team is good. Nobody funded the six weeks of unglamorous data work at the start, because six weeks of data work has never once made a board paper look exciting, and now they're nine months in, spending money at a rate that requires a story, and the story has to be about the model because that's the part anyone can be persuaded to care about.

Let me be specific about what actually goes wrong, because "data quality" as a phrase is so worn out that it no longer transmits anything.

"We have loads of data"

I hear this in the first meeting almost every time, and I've learned to treat it as a warning rather than a reassurance.

Having loads of data means you have loads of rows. It doesn't tell you whether those rows describe the thing you want to predict, whether they were collected consistently, whether the process that generated them is the same process running today, or whether half the useful signal was thrown away by a form redesign in 2021 that nobody thought to mention.

A manufacturer once told me they had eight years of sensor readings. True. What they didn't have was eight years of comparable sensor readings, because the line had been re-instrumented twice and the units changed the second time. Roughly a third of the archive was measuring something subtly different from the rest, and no field in the schema recorded that. It took two engineers eleven days to work this out, mostly by finding a retired maintenance supervisor who remembered.

That's the shape of it. The problem is almost never volume. It's that the meaning of the data changed over time and nothing wrote it down.

Nobody agrees what the label means

This is the one that quietly kills classification projects.

You want a model to flag "high-risk" claims, or "urgent" tickets, or "defective" parts. So you get historical examples labelled by the people who do the job. Fine. Then you take a hundred of those examples, hand them to three experienced people independently, and ask them to label them again from scratch.

Do this. Please do this before you build anything. In my experience, on genuinely hard categories, three experts agree on maybe 60 to 75% of cases, and the disagreements are not random — they're structural. One person thinks "urgent" means the customer is angry, another thinks it means there's a contractual clock running. Both have been labelling tickets for years, and the training set is a blend of two incompatible definitions.

No model can resolve that. It'll learn the blend, produce mush, and you'll blame the architecture. The fix is a couple of days in a room with the people who know, arguing until the definition is written down in a way that survives contact with an edge case. This is the least technical work in the whole project and it has the highest leverage of anything you'll do.

The plumbing nobody owns

Then there's access, which stalls more pilots than any modelling problem I've seen.

The data lives in four systems. One of them is a supplier's, and the contract doesn't clearly permit this use. One of them is exportable only through a report designed for humans, which means a CSV where the header row is a merged cell and the totals are inline. One of them has a nightly job that everybody assumes is fine because it hasn't alerted, and which has been silently dropping records with non-ASCII names since the last upgrade. And the person who understands the fourth one left in March.

None of this is interesting and all of it is real. When a project timeline says "week 1–2: data acquisition," what it actually means is "weeks 1–9, contingent on a legal review nobody has requested yet."

What I'd do instead

Spend the first six weeks on the data and say so out loud, in the plan, with the money attached, before anyone gets excited. Frame it as what it is: the part that determines whether the rest works.

In that time, do four things. Pull a real sample and look at it with your own eyes — actual rows, on a screen, not summary statistics, because summary statistics hide exactly the corruption you're hunting for. Reconstruct the provenance of each field: who enters it, when, under what pressure, and what changed. Run the labelling agreement exercise. And build the smallest end-to-end pipeline you can, moving real data from source to output with nothing clever in the middle, so the access problems surface in week two instead of month five.

If the data doesn't survive that, you've learned it for the price of six weeks rather than nine months, and you can go and fix the collection process, which was probably the real project anyway.

There's a version of this article that ends on something inspiring. I don't have one. The work is dull, it's most of the job, and the teams who do it ship while the teams who skip it hold a lot of steering group meetings about model selection.

Eleven staff, forty thousand case notes

Dimple Paratey — Tue, 24 Sep 2024 00:00:00 +0000

I ended up sitting next to a woman called Ruth at a fundraising thing last spring, the sort of evening where everyone has a lanyard and the wine is warm. She runs a homelessness charity in the Midlands. Eleven staff. Somewhere north of forty thousand case notes going back nineteen years, most of them typed into a system that was modern when Blair was in office.

She asked me, quite directly, whether "all this AI" was something she should be paying attention to or whether it was for organisations with more money than hers. And I found I couldn't answer with any of my usual material, because everything I normally say assumes a company that can afford to be wrong once.

Charities can't. Every pound they spend on a tool that doesn't work is a pound somebody donated believing it would go somewhere else. That changes the maths, and it changes what a responsible recommendation looks like.

Where it actually helps

I went away and did some homework, and then I went back to Ruth. What I found is that the useful stuff in the third sector is almost embarrassingly mundane, which is exactly why it doesn't get written about.

Grant applications. A small charity's fundraising is often one person writing similar-but-not-identical applications over and over, each with its own word limits and its own idea of what an "outcome" is. The source material — what the charity does, who it helps, what happened last year — barely changes. That's a genuinely good fit: drafting from material you already have, with a human who knows the funder doing the final pass. Ruth's fundraiser reckoned it gave her back most of a day a week, and a day a week is a meaningful fraction of an eleven-person organisation.

Reporting. Funders want impact reports. Impact reports are usually a person pulling numbers out of a case management system and writing the same twelve paragraphs in a slightly different order. Same logic applies.

And then the case notes. Nineteen years of them. Ruth's team had a real and rather sad problem: somebody would come through the door for the third time in six years, and unless the worker on duty happened to have been there for all three, that history was effectively invisible. Not lost — sitting right there in the database — just not findable by anyone in the ninety seconds you have before someone stops trusting you. Being able to ask a question of nineteen years of records in plain English is, for that work, not a productivity gain. It's the difference between treating someone as a stranger and treating them as a person you've met before.

Where I'd be careful, and I mean properly careful

That last one is also the one that worries me most, and I told her so.

Case notes about vulnerable people are about as sensitive as data gets. Names, addresses, mental health, immigration status, children, abuse. If any of that ends up in a system where it's being used to train someone's model, or sitting on a server in a jurisdiction nobody checked, the charity has done real harm to people who had very little bargaining power to begin with. That's not a compliance risk, it's a betrayal, and it lands on people who came in asking for help.

So my honest advice to Ruth was: do the grant writing and the reporting first, because the material is public-facing and the downside of getting it wrong is an awkward paragraph. Do the case notes second, slowly, with someone who understands data protection actually in the room, and probably with something that runs on infrastructure you control rather than a free tier that pays for itself with your data.

The other thing I'd flag, gently, is that the sector is being circled at the moment. Charities are being sold AI transformation by consultancies who've noticed that "digital transformation" money exists in the grant landscape. Some of that is fine. Some of it is a discovery phase and a report, funded by a grant, that produces nothing anyone uses. If a proposal doesn't end with a specific person doing a specific task differently on a specific Tuesday, it isn't a project, it's a document.

The bit that stayed with me

What struck me most about the conversation wasn't the technology. It was that Ruth's instinct was already right and she just didn't trust it, because everything written about AI is pitched at organisations fifty times her size and written in a register designed to make her feel behind.

She isn't behind. She has a clear-eyed view of what her staff spend their time on, which is more than most FTSE boards can say. That's the whole prerequisite. The tooling question is downstream of it and much less interesting.

If you're running something small and worthy and you're being told you need an AI strategy — you probably don't. You might need one afternoon, a list of the three things that eat your team's week, and someone honest enough to tell you when the answer is no.

We do that conversation for free for charities, as it happens. Ruth's still going. The case notes project is on hold until they can afford to do it properly, which I think was the right call.

Build, buy, or wait: the third door

Bhaskar Paratey — Tue, 06 Aug 2024 00:00:00 +0000

Every few weeks somebody asks me whether they should build their AI capability in-house or buy it from a vendor. I always answer, because it's a fair question, but I've come to think it's the wrong shape. Build-versus-buy gets framed as a two-way choice mainly because two-way choices fit on a slide and go to a vote cleanly. There are three doors. The third one is "wait," and in my experience it is simultaneously the cheapest option available and the one almost nobody picks, because nobody has ever been promoted for recommending it.

Let me take them in order, since the reasoning is different in each case.

Buying, and actually meaning it

Buy when the problem you have is a problem thousands of other companies also have. Transcription. Document extraction. Support ticket routing. Meeting summaries. There is no strategic advantage waiting for you at the end of building your own transcription service, and a vendor who does nothing else will be better at it than you within a quarter.

The failure mode here isn't buying. It's buying and then refusing to accept what you bought. A team purchases a platform, discovers it does the job 80% the way they'd like, and then spends fourteen months and a small fortune customising the remaining 20% until they've built a bespoke system they can't upgrade, can't support, and don't own the source code to. You've now paid the full cost of building plus a licence fee. If a product doesn't fit your process, the honest options are to change your process or to walk away, and changing the process is usually right and always unpopular.

Three questions I'd want answered before signing anything. Where does our data go, and can we get it back in a usable form if we leave? Which model is under the hood, and what happens the day you swap it for a cheaper one without telling us? And what does the price look like at ten times our current volume — not because we'll get there, but because I want to see whether the pricing was designed by someone who thought about it.

Building, with your eyes open

Building is right when the thing is genuinely yours — when it runs on data nobody else has, encodes judgement specific to your business, or is the actual product you sell. Fair enough. But be clear about what you're signing up for, because the initial build is the smallest line item in the whole undertaking.

What you're really committing to is a second year. The retraining pipeline when the data drifts. The evaluation harness that tells you whether last week's change made things better or quietly worse. The monitoring, the on-call rota, the runbook. The awkward Tuesday when the one engineer who understood the whole thing hands in their notice and you discover the design lived entirely in their head. I've seen more AI systems die of maintenance neglect than of bad modelling. The first version is a sprint; everything after it is a standing cost, and it doesn't stop.

So the test isn't "can we build this." Almost any competent team can build a first version now; the tooling is extraordinary. The test is whether you're prepared to fund it in year three, when it's boring and works and nobody's excited about it any more.

The third door

Wait when the problem is real but the ground is moving faster than your procurement process. This is more common than people admit. If a category of capability is getting roughly twice as good and half as expensive every year or so, a three-year contract signed today locks you to today's economics for the entire period in which those economics are going to change most. Sometimes the correct move is to sit still for six months and buy the thing that doesn't exist yet, cheaper.

The critical part: waiting is not the same as doing nothing, and if you treat it as an excuse to do nothing you've simply lost six months. Waiting well means spending that time on the parts that don't expire. Write down how the process actually works today, in real detail, including the exceptions people handle by instinct and have never documented. Measure the baseline — how long the task takes now, how often it's wrong now — because without that number you will never be able to prove any tool helped. Clean up the data. Fix the permissions mess. Sort out who owns which system.

None of that gets thrown away when you eventually build or buy. All of it is required either way. And most of it is what stalls projects six weeks in, when the pilot grinds to a halt because nobody can get access to the CRM export.

The test, compressed

I ask three things. Is this differentiating — would a competitor with the identical tool beat us anyway on something else? Is the hard part the data, and is that data ours and messy? And how fast is this category moving right now?

Generic problem, mature category: buy it, and don't customise it into a coffin. Differentiating problem, your data, stable enough tooling: build it, and budget for year three. Real problem, category in flux: wait deliberately, and spend the wait on the plumbing.

The companies I've watched get the most out of AI over the last few years weren't the fastest movers. They were the ones who knew which of their problems was which.

Smart cities are easy to sell. Who are they actually for?

Dimple Paratey — Mon, 01 Jul 2024 10:15:00 +0000

There's a man I pass most mornings on the school run. He gets around our high street in a wheelchair, and watching him do it is a daily reminder that the city was designed by people who never had to. The dropped kerb that isn't quite dropped. The "accessible" route that adds twenty minutes. The pavement café that's colonised the only flat bit of path.

I think about him whenever I'm shown a "smart city" pitch, because he is almost never in it. The slides are full of glowing dashboards and sensors and traffic flowing like water, and somewhere in there, unspoken, is a fantasy of a citizen who is young, able-bodied, owns a smartphone, and has nothing to hide. He is not that citizen. Nor, on a bad day, am I.

"Smart city" isn't really a technology. It's a shopping basket — sensors, dashboards, traffic systems, and, far too often, facial recognition — sold to councils as one tidy bundle. Some of what's in the basket would genuinely make life better for the man on the high street. Some of it would make him a data point whether he likes it or not. The whole job is telling those apart before someone signs a fifteen-year contract.

The stuff that genuinely helps real people

Let me start with what I'd happily see my own council buy, because it's worth saying that some of this is properly good.

Traffic signals are the obvious one. Most of them were timed decades ago using rules of thumb, and they've been faithfully causing jams ever since. Adjusting them to what's actually on the road eases congestion and clears the air a bit — and, crucially, it works perfectly well without knowing who's sitting in any of the cars. That distinction matters more than anything else in this piece, so hold onto it.

Then the unglamorous plumbing: predicting which water main or stretch of road is about to fail and fixing it before it floods someone's street. Tuning the heating in schools and hospitals so they stop burning money. Working out where the ambulances should wait so they reach people faster. None of this makes a flashy demo. All of it changes someone's actual day.

And the one closest to my heart — routing that avoids the steps, the broken pavements, the dark underpass. Nobody puts accessibility tech on the conference poster. But for the man on my high street, that's not a feature. That's whether he can get to the shops alone.

Notice what every one of those has in common. A clear job, a saving you can point at, and no need to know who anyone is.

The stuff I'd fight before it's installed

Now the basket gets darker, and the danger is that the cost never shows up on the day you buy it. It shows up years later, on someone else.

The camera that smooths the traffic can run facial recognition with a software update. The sensor listening for noise can pick out an individual. Once that hardware is bolted to the lampposts, deciding not to use it that way stops being a setting and becomes a political brawl — one that residents usually lose. You have to draw that line before the kit goes up, because afterwards the kit decides.

Vendor lock-in is the boring trap that bites hardest. Sign with the wrong supplier and you can lose the right to switch, to open your own data, or to change your mind. That's not a software preference. That's handing a private company a chunk of your city's nervous system.

And then the systems that decide where to send police patrols or which homes to inspect — trained on yesterday's data, they faithfully reproduce yesterday's unfairness, aimed at the same communities that have always been over-policed and under-served. The bias isn't in anyone's intentions. It's baked into the history the model learned from, which is exactly why it keeps happening.

The one that makes me angriest is exclusion dressed as progress. A service that requires an app, or a digital ID, or being online — that's a service that has decided the elderly, the homeless, and the newly arrived don't count. A city that locks people out of the basics to look modern has failed at the only thing a city is for.

What I'd write into the contract, in ink

Some places have already worked out the right defaults, and I'd shamelessly copy them. Amsterdam publishes a register of what each of its algorithms does and how you can challenge it. Estonia lets residents see who looked at their data and why. Copenhagen shares back what the city collects. Barcelona uses tech to summarise what citizens say, not to replace asking them.

Translate all that into plain contract language and it comes out as a short, stubborn list. Every system that makes decisions about residents is documented publicly. Collect the least data, keep it the shortest time, don't stitch datasets together on a whim. No facial recognition in public spaces — full stop, no asterisk. The city owns its own data. And you ask people before you build, not after, in a way that can actually change the answer.

So here's the test I'd apply to anything in that basket, on behalf of the man on the high street. Does it need to identify individuals to do its job? And if it goes wrong or gets breached, who pays, and can it be undone? The adaptive traffic lights pass both, easily. Public facial recognition fails both, badly. Most things sit in between — and that uncomfortable middle is precisely where someone has to fight for the right clauses.

The best cities I've seen aren't the ones bristling with the most sensors. They're the ones that decided who they were building for, and what they'd refuse to build, before a salesperson decided it for them.