Model risk management: what it is, the framework, and how it applies to AI

Model risk is the chance of loss because a model is wrong, or because a correct model is used for something it was not built for. The US banking agencies’ current guidance, SR 26-2, defines it as “the potential for adverse financial consequences associated with models”, which may come from decisions based on model output ^[1]. Its 2011 predecessor, SR 11-7, used a wider definition that also named poor business decisions and damage to reputation ^[2].

What counts as a model matters as much. SR 26-2 defines a model as a complex quantitative method, system or approach that applies statistical, economic or financial theories to turn input data into estimates, and excludes simple spreadsheet arithmetic and deterministic rule-based software ^[1]. Canada’s OSFI uses a broader definition that explicitly includes “AI/ML methods”, and describes every model as having an input, a processing and a result component ^[4].

The best-known failure is JPMorgan Chase’s “London Whale”: a credit derivatives portfolio run by the bank’s Chief Investment Office lost more than $6 billion, and the value-at-risk model used to measure it “was badly flawed, understating risk”, as Ben W. Heineman, Jr. summarized the findings of the bank’s own task force report on the Harvard Law School Forum on Corporate Governance ^[6].

Model risk management is the discipline of controlling that risk across a model’s life: building it soundly, checking it independently, using it within its limits, watching how it performs and retiring it cleanly. The US and Canadian texts scale the effort to the model. SR 26-2 says higher-materiality models need more comprehensive and rigorous oversight, and that materiality comes from a model’s purpose together with its exposure ^[1]. OSFI applies E-23 in proportion to an institution’s size, strategy, risk profile, complexity and interconnectedness ^[4].

The idea at the center of MRM is effective challenge: critical analysis by objective experts who assess model risk and bring about appropriate changes, carried out by people with the expertise, independence and standing to make changes happen ^[1]. If validators can’t force a fix, the program is paperwork.

MRM grew up in banking, but the same practices fit any organization whose decisions depend on models: insurers pricing policies, lenders scoring applicants, hospitals triaging patients or employers screening candidates. Outside regulated finance it is usually voluntary, and teams often borrow the bank playbook alongside the NIST AI RMF, which NIST released in January 2023 “for voluntary use” ^[7].

Regulators organize MRM in slightly different ways. The US guidance covers development and use, validation and monitoring, governance and controls, and third-party models ^[3]. OSFI names the lifecycle stages as design, review, deployment, monitoring and decommission, and notes they are not always sequential ^[4]. The PRA’s five principles are model identification and risk classification, governance, development and use, independent validation, and risk mitigants ^[5]. Combined, a working framework looks like this:

1. 01

   Identify models and build the inventory

   Decide what is a model under your definition and record each one. SR 26-2 says an effective inventory holds enough information to understand model risks ^[1]; OSFI calls it an enterprise-level system of record, kept current, that keeps recently decommissioned models for a time ^[4].

2. 02

   Rate and tier each model

   Assign a risk rating that drives everything downstream. OSFI combines quantitative factors such as financial impact with qualitative ones such as complexity, autonomy and customer impact, and uses the rating to set review frequency, documentation, approval authority and monitoring ^[4].

3. 03

   Develop and document

   Build with testing as part of development, and document purpose, design, data and limitations. SR 26-2 treats adequate documentation as support for effective model risk management ^[1].

4. 04

   Validate independently

   Assess conceptual soundness (the design), outcomes analysis (outputs against real-world results) and ongoing monitoring ^[1]. Validation should be independent of development; the PRA makes independent validation a principle of its own ^[5].

5. 05

   Approve and set limits on use

   Approve each model for a stated use, with conditions. Effective use depends on understanding a model’s limitations, and controls can include limits on use or closer monitoring ^[1].

6. 06

   Monitor in production

   Check that the model keeps performing as expected over time ^[1], with thresholds that trigger review. For machine learning this includes data and prediction drift.

7. 07

   Govern vendor models the same way

   Validate vendor products, by internal or outside parties ^[1]. OSFI requires third-party models to be identified, inventoried and rated on a standalone basis ^[4].

8. 08

   Decommission and keep records

   Retire models formally and keep them in the inventory for a reasonable period ^[4]. Track findings, exceptions and remediation, which SR 26-2 lists as part of documentation ^[1].

Most banks assign MRM roles using the three lines of defense. The Institute of Internal Auditors updated the idea in July 2020 as the Three Lines Model, replacing the older “Three Lines of Defense” name ^[8]. SR 26-2 itself does not use the phrase, but it describes the same roles ^[1].

A model risk management policy turns the framework into rules people can follow and auditors can test. This checklist lists the sections a policy usually needs, with the regulatory text each one answers to.

The practical upshot for a bank operating in all three countries: one inventory and one tiering scheme can serve all three, but the US definition of a model is now the narrowest, so the Canadian and UK programs will hold models, including AI, that the US program can exclude ^[1][4][5].

Traditional machine learning fits MRM well. SR 26-2 applies its principles to traditional statistical and quantitative models and to AI models that are neither generative nor agentic ^[1], so a gradient-boosted credit model gets the same inventory, validation and monitoring as a logistic regression. What changes is the work inside each step: explainability testing during validation, fairness testing by group, drift monitoring in production, and documentation of training data. OSFI also names autonomy as a rating factor and expects staff with the skills for novel technologies such as AI ^[4].

For LLMs and agents, most firms run a parallel track: keep them in an AI inventory even if they leave the model inventory, tier them by use, and test them before release with scenario-based or “black-box” evaluation. Monitaur, for example, describes pre-deployment synthetic scenarios that probe prompt injection, jailbreaks, toxicity and bias for generative and agentic systems ^[10]. NIST’s AI RMF, which NIST says is being revised under the White House AI Action Plan, is the usual reference framework ^[7]. Our AI agent governance guide covers controls specific to agents, and our AI bias guide covers fairness metrics.

MRM platforms automate the inventory, validation workflows, documentation and monitoring. These are examples from our directory, described from their own pages on November 5, 2026; we have not tested them. For a ranked comparison, see AI model risk management software.

1. 01

   ValidMind

   Best for: Banks that want a published SR 26-2 mapping

   Its SR 26-2 implementation page maps features to six areas: telling models from non-models, materiality tiering, ongoing monitoring and outcomes analysis, development documentation, independent validation and governance. It notes that SR 26-2 excludes generative and agentic AI.^[9]

   ValidMind profile →

2. 02

   ModelOp

   Best for: One inventory for ML, generative AI and vendor AI

   Describes its Inventory as a system of record covering ML models, generative and agentic AI, and third-party AI including AI embedded in SaaS, with intake, rules-based risk tiering and evidence collection. Its inventory page lists SR 11-7 and the EU AI Act but not SR 26-2.^[11]

   ModelOp profile →

3. 03

   Monitaur

   Best for: Validating generative and agentic systems

   Says its FlightSim service runs black-box, pre-deployment scenarios against internal or vendor generative and agentic systems, and that its control library draws on the EU AI Act, NIST AI RMF and state algorithmic discrimination laws.^[10]

   Monitaur profile →

4. 04

   DataRobot

   Best for: Generating model documentation

   Its compliance documentation feature generates editable model documentation, downloadable as a Word file, that summarizes model components. DataRobot says it can help banks with SR 11-7; the page did not mention SR 26-2.^[12]

   DataRobot profile →

5. 05

   Fiddler AI

   Best for: Monitoring predictive models and agents

   Its governance page names SR 26-2 among supported regulations and describes monitoring for drift, bias and performance degradation, audit evidence, custom reports for MRM reviews and an agent registry.^[13]

   Fiddler AI profile →

• SR 11-7 and SR 26-2 →The US bank guidance line by line, and what banks should do now.
• OSFI E-23 explained →Canada’s final model risk guideline for banks and insurers.
• AI model risk management software →MRM platforms ranked for banks and insurers.
• How to audit AI →Audit types, the audit process and the evidence auditors ask for.
• NIST AI RMF →The voluntary US framework used for generative AI risk.