Gary Cheung · 20 February 2026 · 5-minute read
At a glance
- The risk: Architecture and workflow choices can give AI influence or authority that a later review struggles to constrain.
- The governance implication: Define permitted actions, information boundaries, evidence and stop authority before committing to the design.
- The leadership takeaway: Ask whether the proposed design makes meaningful challenge and intervention possible before approving delivery.
Before a bank builds an AI assistant for account servicing, it must decide whether the system may recommend a fee adjustment or execute it. That choice determines whose money can be affected, which permissions are needed, and whether a human can intervene before the action takes effect.
This is a governance decision with architectural consequences. If risk review begins only after the workflow is built, leaders may face an expensive redesign or pressure to accept the existing boundary. The practical question is who owns the customer outcome and who can challenge or stop the process while design options remain open.
Late review still matters. The problem is relying on it to supply constraints that should have shaped the system from the beginning.
Design choices distribute authority and exposure
Model selection is only one part of the decision. Data access determines what information the system can expose. Tool permissions determine what it can change. Workflow placement determines whether staff treat an output as a suggestion, a default, or an instruction.
Consider an illustrative bank assistant, not a client case. A design that produces a recommendation for a trained employee differs materially from one that updates account records automatically. Both can cause harm, but the second creates a shorter path from an erroneous output to a customer consequence.
The business owner should approve the intended use and acceptable consequences. Technology teams should translate that boundary into permissions, interfaces and recovery options. Risk and control functions need authority to challenge the assumptions. A committee endorsement without these decisions leaves implementation teams to make risk choices through technical defaults.
Existing information-risk and application-security disciplines remain relevant: identify sensitive information flows, restrict access and action rights, and test whether controls enforce the approved boundary. AI-specific analysis adds how uncertain outputs and human reliance affect that boundary.
Oversight must be technically possible
A policy requiring human review is useful only if the workflow gives the reviewer enough evidence, time and authority to act. A person clicking approve on a fluent recommendation may have little opportunity to identify an omitted fact or an unsupported inference.
Design should make the relevant source information available, identify the action awaiting approval, and allow escalation before the consequence occurs. Reviewers need a workable alternative when evidence is insufficient. Staffing and response time must fit the volume and urgency of the decisions.
Stop authority also needs an implementation path. Disabling a model endpoint may not stop queued actions, connected tools or an operational process that already relies on its outputs. Specify what is paused, who can initiate it, and how affected work continues safely.
Logging should support reconstruction of the decision: relevant inputs, outputs, versions, approvals and actions, subject to applicable information-handling requirements. These records do not guarantee accountability, but their absence can make challenge and incident analysis much harder.
Frameworks support early governance; leaders operationalize it
The NIST AI Risk Management Framework 1.0, released on 26 January 2023, is voluntary guidance and is under revision. Its MAP function connects contextual understanding to an initial go/no-go decision. MAP 1.6 addresses system requirements and socio-technical design; MAP 3.5 addresses human oversight. It does not confine governance to a final review.
The OECD AI Principles likewise connect accountability to actors’ roles and call for traceability and ongoing risk management across the lifecycle. They are policy principles, not a universal legal approval rule.
ISO/IEC 42001, published in December 2023, specifies requirements for establishing, implementing, maintaining and continually improving an AI management system. A management-system standard should not be treated as proof that a particular deployment is safe or meets every applicable legal obligation.
The leadership task is to turn these foundations into local decision rights. Which commitments require review? What evidence is necessary? Who can require a different design? Citing a framework cannot answer those organization-specific questions by itself.
Govern commitments, then test the implemented boundary
Place governance at commitments that materially change exposure: connecting customer data, granting a tool permission to act, increasing automation, or making a workflow dependent on AI. A proportionate early review can approve a bounded design while identifying questions that must be resolved before live use.
For the illustrative bank assistant, that could mean allowing recommendations using approved test data while withholding permission to alter accounts. Moving to live records or executable fee adjustments would require a separate decision supported by evidence about access, performance, human intervention and recovery.
Keep implementation review and deployment testing. An approved design can be built incorrectly; a control that appears adequate on paper can fail in the workflow. After deployment, changes in models, data, permissions or business reliance may require renewed approval.
Commercial deadlines and sunk cost should not silently become risk acceptance. Record unresolved issues, the accountable owner, and who can withhold approval, restrict use or require redesign. Early governance preserves options; later assurance checks whether those options became working controls.
Questions for Leaders
- Which design choice gives AI consequential influence or permission to act, and who approves that boundary?
- Can a reviewer obtain the evidence, challenge the recommendation and stop the action before harm occurs?
- Which changes in data access, permissions or business reliance require renewed approval, and who can withhold it?
Closing Thought
Governance belongs wherever a business commits to an AI-supported decision. Bringing it into design makes accountability and intervention requirements actionable. Keeping it through testing and operation establishes whether the implemented process remains within the approved boundary.
Sources and Further Reading
- NIST AI RMF 1.0 Core — MAP, GOVERN and MANAGE guidance.
- NIST AI Risk Management Framework overview — voluntary status and revision notice.
- OECD AI Principle 1.5: Accountability — roles, traceability and lifecycle risk management.
- ISO/IEC 42001:2023 — official AI management-system standard overview.
- AI Governance That Arrives Too Late: When Frameworks Exist but Risk Still Escalates.
- From Guardrails to Governance: What Actually Changes When Organizations Get Serious About AI Risk.
About Gary Cheung
Gary Cheung writes about AI governance, information risk, and accountable technology decisions. To discuss an AI governance challenge, share the decision context through the AI Decision Review.
