Accountability when the decision was made by a system

Identify the decision chain

When an automated system denies credit, ranks a candidate, prices risk or flags suspected fraud, several actors may influence the result. The developer designed the model, a provider supplied it, the deployer configured thresholds, data sources shaped the output and a human may have accepted it without meaningful review. Liability analysis must reconstruct that chain rather than treat the system as a single black box.

The applicable legal duties may arise from contract, negligence, employment, consumer protection, discrimination, data protection, sector regulation or professional responsibility. A technical error is relevant only when connected to a legal duty, causation and recoverable harm.

Evidence must be designed into the system

After a disputed decision, organisations may discover that they did not retain the model version, input data, prompt, rule configuration, confidence score or human-review record. Without those materials, they cannot explain the decision or test whether the same facts would produce the same result.

Governance should require logs, change control, validation, escalation and a record of meaningful human intervention. Contracts should allocate access to evidence, audit cooperation, incident response, intellectual-property limits and responsibility for regulatory change.

A practical accountability model

Responsibility should follow control. The party choosing the purpose, data, threshold or action may carry a different duty from the model developer. Clear role mapping, impact assessment and documented supervision make both compliance and dispute defence stronger, particularly where the technology operates across jurisdictions.