The Responsibility Gap: When AI Advises, Who Owns the Decision?

Slug: responsibility-gap-ai-decisions
Tags: AI, Philosophy, Human Oversight
Meta description: Explore the responsibility gap in AI-assisted decisions and why human oversight must include authority, evidence, traceability and answerability.

Featured image: AI-generated conceptual illustration, not a photograph of a real decision-making process.

An AI system recommends that a loan be refused, an applicant be shortlisted, a patient be prioritised or a public resource be allocated elsewhere. A human sees the recommendation and clicks approve. Who made the decision?

The convenient answer is “the human”. The equally convenient excuse is “the algorithm”. Both can conceal how responsibility actually moved through the system.

The philosophical problem is not merely whether artificial intelligence can produce a decision-like output. It is whether a person or institution can remain answerable when the system has shaped what they noticed, which options appeared reasonable and how quickly they were expected to act.

The responsibility gap is a design problem

Philosopher Mark Coeckelbergh’s open-access paper Artificial Intelligence, Responsibility Attribution, and a Relational Justification of Explainability examines responsibility through questions of control and knowledge. The paper also shifts attention towards answerability: people affected by AI-assisted decisions may reasonably demand reasons from those who made or authorised them.

This matters because saying that a human was “in the loop” proves very little. The person may have lacked time, training, relevant evidence or authority to challenge the output. Oversight can be present in an organisational chart while absent at the moment it matters.

A responsibility gap therefore appears when an organisation gains the practical power of automated recommendation but cannot identify a capable, informed and answerable decision owner. The machine becomes a place where causal influence accumulates while moral language becomes strangely vague.

An output is not yet a judgement

AI systems can classify, rank, predict, generate and recommend. Those capabilities may be useful. Yet a recommendation does not decide what ought to matter in a particular situation.

A model can estimate a probability from available data. It does not thereby settle whether the data are appropriate, whether an exception deserves attention, whether the consequences are proportionate or whether the institution’s objective is defensible. These are normative questions: they involve values, duties and the treatment of people.

Human judgement is not valuable because humans are automatically wiser or less biased. Human decision-makers can be inconsistent, prejudiced or careless. The point is institutional: people can be assigned duties, required to give reasons, challenged by those affected and subjected to remedies or sanctions. An output cannot attend an appeal hearing, apologise meaningfully or repair an injustice.

Oversight must be more than a ceremonial click

Article 14 of the EU Artificial Intelligence Act requires high-risk AI systems to be designed so that natural persons can effectively oversee them during use. The provision addresses capabilities such as understanding limitations, monitoring operation, recognising automation bias, interpreting outputs and intervening or stopping a system.

That legal framework applies to defined systems and actors; this article is philosophical analysis, not legal advice. Still, it reveals an important principle: oversight is meaningful only when the overseer can understand enough, act in time and change the outcome.

A reviewer who is punished for disagreeing with the model is not exercising independent judgement. A reviewer presented with hundreds of cases and seconds per case is not providing careful scrutiny. A reviewer who never sees uncertainty, missing data or alternative explanations is being asked to endorse, not oversee.

Five tests for responsible human judgement

Before describing a process as human-led, an organisation should be able to answer five questions.

• Authority: Can the person reject, pause or modify the AI recommendation without needing impossible approval?
• Competence: Do they understand the domain, the system’s intended use and its important limitations?
• Evidence: Can they inspect the information supporting the recommendation, including relevant uncertainty and missing inputs?
• Time: Do they have a realistic opportunity to think, seek another view and identify exceptional cases?
• Answerability: Can the institution explain the final decision to the affected person and provide a route for challenge or remedy?

If several answers are no, the phrase “human in the loop” may be organisational theatre. Responsibility has not disappeared, but the system may have made it difficult to locate and exercise.

Responsibility is distributed—but it must not be diluted

AI decisions are rarely the product of one individual. Developers choose objectives and constraints. Data teams assemble training or evaluation material. Vendors describe capabilities. Procurement teams select products. Managers define workflows and performance targets. Front-line staff apply recommendations. Leaders decide what happens when harm is reported.

This is distributed responsibility. It does not mean that everyone is equally responsible, nor that nobody is. Different actors control different risks and should carry duties proportionate to their role.

The NIST AI Risk Management Framework treats governance as a cross-cutting function for managing AI risk. The OECD AI Principles, updated in May 2024, likewise include accountability among their values-based principles. These frameworks are not substitutes for sector-specific law, but they support the idea that responsibility must be organised across the lifecycle rather than deposited on the final user.

A sensible record should therefore show which system version was used, what information was supplied, what recommendation appeared, who reviewed it, whether it was changed, and why the final decision was made. Traceability does not solve every ethical problem, but without it accountability can collapse into guesswork.

The person affected is not an edge case

Technical governance often begins with the organisation: accuracy, reliability, compliance and operational risk. A relational view begins elsewhere too—with the person who must live with the outcome.

That person may need to know whether AI materially influenced the decision, which factors were decisive, how errors can be corrected and who has power to reconsider the case. Explainability is not only a debugging feature. It is part of treating someone as a participant in a moral and institutional relationship rather than merely as an object of classification.

Not every system can reveal every internal operation in simple language. But an institution can still explain its purpose, relevant inputs, known limitations, review process and grounds for the final decision. “The model said so” is not an explanation; it is evidence that the organisation has confused computation with justification.

A practical responsibility ledger

For any consequential AI-assisted workflow, create a one-page responsibility ledger before deployment. Name the system owner, the decision owner, the technical maintainer, the person responsible for monitoring harms and the route of appeal. Record what each role can actually do.

Then run a reversal test: if the system produces a confident but harmful recommendation tomorrow, who can detect it, who can stop it, who must explain it and who must repair the consequences? If those answers are unclear, the governance design is unfinished.

The deeper philosophical lesson

Artificial intelligence forces us to separate influence from responsibility. A system may strongly influence a decision without becoming the kind of being we can meaningfully praise, blame or demand an account from. Humans and institutions may retain responsibility even when no single person controlled every technical step.

That is uncomfortable because automation promises efficiency partly by compressing chains of judgement. But moral responsibility cannot be compressed into a button labelled approve.

The goal is not to keep a token human beside every machine. It is to preserve the real conditions of responsible judgement: knowledge, authority, time, traceability and answerability to the people whose lives are affected. When those conditions disappear, the problem is not that the AI has become responsible. It is that humans have built a decision system in which responsibility has nowhere effective to stand.