Submission: consultation on automated decision-making in public administration
Our full response to a public consultation, published as submitted. We supported three proposals, opposed one, and identified an omission we consider more consequential than any of them.
We publish our consultation submissions in full, including where our advice was not taken. This is our complete response as submitted.
Summary of position
We support the proposals on registration, notification and appeal. We oppose the proposed exemption for systems described as decision-support rather than decision-making. And we consider the absence of any capacity provision for the supervising authority to be the most consequential feature of the draft.
On the decision-support exemption
The draft exempts systems that inform rather than determine a decision. We understand the intention and we think the distinction will not survive contact with practice.
A recommendation presented to an official who has forty seconds per case, no ready means of checking it, and a throughput target that disagreement jeopardises, is a decision in every respect that matters to the person affected. The formal presence of a human does not alter the outcome, and the exemption will be relied upon precisely by the systems whose effect is greatest.
We suggest the test should be whether the recommendation is in practice followed, not whether a person is formally entitled to depart from it.
On the omission
The draft creates obligations of registration, assessment, notification and appeal, all to be supervised by an authority for which it provides no additional technical staff, no funding and no power to set its own pay scales.
We have set out elsewhere why this is the binding constraint on enforcement. Obligations administered by a body that cannot assess compliance produce a compliance industry rather than compliance. We would rather see three obligations that are enforced than nine that are recorded.
On appeal
We support the appeal provisions and suggest one addition: that the appeal reach a person empowered to reverse the decision, and that this be stated. Appeal routes that terminate in a review of whether the process was followed are common, and they are not appeals.
