By Gary Cheung · 5-minute read
At a glance
- The risk: Traditional control testing can miss AI-specific exposure, changing behavior, untracked use, and decisions shaped by unreliable outputs.
- The governance implication: Assurance must follow the deployed use case and test data, model behavior, vendors, security, human review, and change over time.
- The leadership takeaway: Do not accept an audit finding without a named owner, an evidence threshold, an escalation route, and authority to restrict or stop reliance.
Organizations are placing AI inside workflows that can affect customers, employees, security, and business decisions. Traditional IT controls still matter, but an audit can test access, change management, logging, and vendor governance without establishing whether an AI-influenced decision is dependable.
The consequential decision is whether leaders can rely on the process after assurance work is complete. The AI audit gap is the difference between the AI uses an organization has deployed and the evidence its governance and assurance processes can actually provide.
Why Traditional Assurance Misses AI-Specific Exposure
Conventional audits usually assess whether a control exists and operates as designed. AI adds uncertainty because system behavior can depend on prompts, retrieved content, model versions, data, integrations, user judgment, and vendor changes that occur outside a familiar release cycle.
The audit scope therefore needs to extend beyond the model. It should include the full decision path: the business purpose, input data, connected tools, output handling, human review, exception process, and downstream action. If any of those elements can materially change the outcome, they belong in the assurance boundary.
Audit the Deployed Decision, Not the Model Label
An “AI-enabled” label does not identify the risk. Auditors should start with the business decision or control that the capability influences. Is it prioritizing a security alert, recommending a customer outcome, drafting regulated communication, or taking an automated action? The answer determines the evidence required and the consequence of failure.
The practical governance test is whether the organization can explain what it is relying on, under which operating conditions, and with what fallback. Evidence should be tied to that use case, not borrowed from a vendor benchmark or a general model description. The accountable owner must also be able to challenge the evidence and reduce or stop reliance when performance, context, or risk changes.
Four Evidence Areas an AI Audit Should Test
An AI-aware audit should test at least four connected evidence areas:
- Security and system boundaries: Examine direct and indirect prompt injection, tool permissions, connected data sources, output handling, and whether the application can perform actions beyond the user’s intended authority. NIST’s Generative AI Profile identifies prompt injection as an information-security risk and recommends testing appropriate to the deployed context.
- Data and contractual controls: Verify what information the service processes, retains, exposes to personnel or subprocessors, and uses for model improvement. Do not treat model training, operational retention, and unauthorized disclosure as the same risk. For example, OpenAI states that business and API data are not used for training by default, while separate retention and account controls still apply.
- Performance and change: Record relevant model, prompt, configuration, and integration versions; define representative tests; monitor outcomes; and reassess after material changes or incidents. A control that worked at launch is not permanent evidence.
- Inventory and human control: Include embedded vendor features and employee-selected tools where they create material exposure. Identify who reviews outputs, handles exceptions, accepts residual risk, and can disable the capability.
A Public Failure Shows the Accountability Gap
Verified public example: In Moffatt v. Air Canada (2024 BCCRT 149), the British Columbia Civil Resolution Tribunal held Air Canada responsible after its website chatbot provided inaccurate bereavement-fare information. The case is specific to its facts and jurisdiction, but it exposes a broader governance point: an organization cannot separate accountability for a customer-facing outcome from the automated channel that produced it.
For auditors, the question is not simply whether the chatbot was tested. It is whether authoritative information, monitoring, correction routes, and accountable ownership were sufficient for the decision the organization allowed the system to influence.
Turn Audit Findings Into Governance Action
Findings should lead to a decision, not another undifferentiated checklist. For each material gap, record the affected use case, possible business consequence, evidence missing, interim safeguard, accountable owner, remediation date, and escalation threshold. Where the evidence is insufficient, leaders should decide whether to limit the use, add human review, change the control design, or stop reliance.
Regulation and frameworks should be treated accurately. The EU AI Act creates binding duties for defined providers and deployers based on the system, role, risk classification, and staged application timetable. The NIST AI Risk Management Framework and Generative AI Profile are voluntary resources that can structure risk work; they do not certify that a specific deployment is safe, compliant, or effective.
The strongest assurance cycle is continuous: inventory the use, test the decision path, monitor change, assign remediation, and return material residual risk to the leader authorized to accept or reject it.
Leadership Questions
- Which AI-influenced decisions could materially affect customers, employees, security, or operations, and are all of them inside the audit scope?
- What evidence demonstrates that controls work under the deployed conditions—not only in a vendor or pre-production test?
- Who owns unresolved findings, and who has authority to restrict or stop reliance before harm occurs?
Closing Thought
An AI audit closes the gap only when evidence changes a decision. The measure of assurance is not the number of controls reviewed; it is whether leaders know what remains uncertain, who owns it, and what must happen next.
Sources and Further Reading
- NIST AI Risk Management Framework
- NIST AI RMF: Generative Artificial Intelligence Profile (July 2024)
- OpenAI enterprise privacy
- OpenAI API data controls
- Moffatt v. Air Canada, 2024 BCCRT 149
- European Commission: AI Act overview
- Beyond1n0: Secure AI Deployment
- Beyond1n0: AI Risk Registers
About Gary Cheung
Gary writes about information risk, AI governance, and executive accountability. Discuss an AI decision or governance challenge.
