Artificial Intelligence

Systems that infer, predict and generate are being placed at the centre of decisions that were previously made by people who could be asked to explain themselves. A loan is declined, a scan is triaged, an application is ranked, a claim is flagged — and the reasoning now sits inside a statistical model rather than in the mind of someone who can be questioned about it. This is not a future prospect. It is the ordinary condition of most large institutions today.

Our concern is less with the capability than with the transfer. When judgement moves from a person to a system, several things travel with it that are rarely accounted for: the ability to notice that a case is unusual, the discretion to depart from procedure when departing is right, the memory of having been wrong before, and the simple fact of someone being answerable. A model can be more accurate than the person it replaces and still leave the institution less capable of recognising its own errors, because the error is now distributed across a training set rather than located in a decision someone made.

We work on the conditions under which such a transfer is defensible. Demonstrated competence at the specific task, in the specific population, rather than a benchmark achieved elsewhere. An accessible route to challenge a result, that reaches someone empowered to reverse it. A named party who remains answerable after deployment, not only during procurement. Honesty about the cases the system was never evaluated on — which is usually a larger set than the cases it was. And a stated threshold at which the institution would stop using it, agreed before the system is relied upon rather than after something has gone wrong.