Who is accountable when an AI system gets it wrong

Updated 17 October 2026

An AI system recommends a supplier. An employee accepts the recommendation. The supplier fails a check that the organisation normally requires. Who is responsible for explaining the decision?

The answer becomes difficult when the business assumes technology owns the system, technology assumes the vendor owns its quality and the vendor's contract leaves important responsibilities with the customer.

Governance should resolve those gaps before the system becomes part of daily work. The organisation needs to know how it controls the use and answers for its consequences.

Assign responsibility for the business use

A named business owner should understand why the system is being used, what performance is acceptable and how failures will be handled. Technical teams need clear responsibility for operation, access and changes within their remit. Relevant control functions and specialists should review the risks that concern them.

These responsibilities can sit within existing structures. A new committee is useful only if it has a clear decision to make and the authority to make it. Approving the same use in several forums can obscure accountability further.

For material uses, leadership should know who may approve deployment, expand the scope and suspend operation. Agree how those decisions will be recorded.

Define what people may rely on

A tool drafting an internal meeting summary creates different consequences from one influencing a payment or a customer decision. Oversight should reflect the actual use, the information involved and the harm an error could cause.

Specify which outputs require checking and what a meaningful check involves. A person asked to review an output needs enough information, time and authority to reject it. An approval button provides little assurance if the reviewer cannot inspect the basis of the recommendation.

Employees also need a route for cases they cannot resolve. Training should include examples of when to pause and ask for help, rather than only demonstrations of successful use.

Make failure manageable

Before launch, agree how users report a suspected error, who investigates and what happens to affected work. Establish whether the organisation can identify relevant outputs and decisions without collecting unnecessary information.

The ability to suspend a system deserves an operational test. Who can do it outside normal working hours? What process takes over? How will customers or colleagues be informed if an issue affects them?

Include providers in this discussion. Ask how changes are communicated, what evidence is available and how incidents are escalated. A dependency remains the organisation's concern even when another party operates the technology.

Review the use as it changes

A system may acquire new users, data connections or permissions after its original approval. Those changes can alter the risk. Define which changes require another review and check performance during ongoing use.

The NIST AI Risk Management Framework provides a voluntary reference for organising risk work across an AI system's life. It supports a continuing process rather than a single approval exercise. [1]

At the next AI review, ask the sponsor to describe one plausible failure and walk through the response. Any uncertainty about who acts is a practical issue to resolve before expanding use.

[1] NIST AI Risk Management Framework Read the source