Guides
NIST AI RMF and OSFI E-23
What evidence Oloproof records toward the NIST AI RMF's Measure function and OSFI's Guideline E-23, item by item, and what it does not record. The editions are the NIST AI Risk Management Framework 1.0 (NIST AI 100-1, January 2023) and Guideline E-23, Model Risk Management (final 11 September 2025, in effect from 1 May 2027 for federally regulated financial institutions in Canada).
This is not certification, not an assessment of your compliance and not legal advice. A framework asks for practices across an organisation; Oloproof produces evidence about AI systems, which is one part of those practices. NIST has said the AI RMF 1.0 is being updated, so check the edition you are held to.
How to read the tables
- Recorded: Oloproof records evidence for the item as written, in every run or record it names.
- Partly: Oloproof records part of what the item asks; the rest is said beside it.
- Not covered: the item asks for something Oloproof does not do.
The items are paraphrased. Read the framework's own text before relying on a row.
NIST AI RMF 1.0: Measure
| Item | What it asks, in short | What Oloproof records | Status |
|---|---|---|---|
| MEASURE 1.1 | Metrics chosen for the risks the Map function found | Each run's metric specifications, rules and release policy, by digest. Choosing them is yours | Partly |
| MEASURE 1.2 | Metrics and controls reassessed over time | Trend marks where the judge or the cases changed; a decision says when its evidence is stale (Gating) | Partly |
| MEASURE 1.3 | Assessment by people who did not build the system | A workspace that decides on its own evidence, labellers counted by account, and a decision record a second party checks offline. Who those people are is yours to arrange | Partly |
| MEASURE 2.1 | Test sets, metrics and tools documented | Every run names its suite, system and evaluator versions, metrics and methods by content address, and the analysis version | Recorded |
| MEASURE 2.2 | Human subject protections | Nothing | Not covered |
| MEASURE 2.3 | Performance measured and demonstrated | Every rule decided on an interval: PASS, FAIL, INSUFFICIENT_EVIDENCE or MANUAL_REVIEW. Whether the suite resembles deployment is yours | Partly |
| MEASURE 2.4 | Behaviour monitored in production | People review a sample the workspace draws from real traffic (Evaluating production) | Partly |
| MEASURE 2.5 | Validity and reliability demonstrated | Intervals with their denominators, and judges measured against people before they may decide (Judges) | Partly |
| MEASURE 2.6 | Safety risks evaluated regularly | Ceilings on what goes wrong, decided on your cases, and a red-team tool's grades read when you have them (Risk ceilings). Oloproof writes no attacks | Partly |
| MEASURE 2.7 | Security and resilience evaluated | Only through ceilings on cases you write or import | Partly |
| MEASURE 2.8 | Transparency and accountability examined | Sign-offs and approvals by two named people, kept in a chain and carried in the decision record | Partly |
| MEASURE 2.9 | The model explained and its output interpreted | Diagnoses by intervention name the part implicated in failures; they explain no model | Partly |
| MEASURE 2.10 | Privacy risk examined | The egress policy keeps content on your side; no privacy risk is assessed | Not covered |
| MEASURE 2.11 | Fairness and bias evaluated | Declared slices measured with multiplicity control; what makes a slice fair is yours | Partly |
| MEASURE 2.12 | Environmental impact assessed | Cost is recorded; energy is not | Not covered |
| MEASURE 2.13 | The measurement itself evaluated | Judges validated against people's labels, and each statistical method admitted by a note, a coverage grid and an audit | Partly |
| MEASURE 3.1 | Risks tracked over time | Trend, freshness and production review | Partly |
| MEASURE 3.2 | Risks that are hard to measure | A rule the evidence cannot decide says so (INSUFFICIENT_EVIDENCE, MANUAL_REVIEW), and an undecided interval says how many more cases would decide it | Partly |
| MEASURE 3.3 | Feedback from end users | Nothing | Not covered |
| MEASURE 4.1 | Measurement tied to the deployment context | Labels from your own reviewers; the context is yours | Partly |
| MEASURE 4.2 | Results informed by domain experts | Reviewers' labels decide what a judge is worth | Partly |
| MEASURE 4.3 | Improvements and declines documented | Comparisons of a change against what is live, paired, with intervals (Compare) | Recorded |
OSFI Guideline E-23 (2027)
| Principle | What it asks, in short | What Oloproof records | Status |
|---|---|---|---|
| 1.1 to 1.3 | Reporting lines, a model risk framework, models fit for their purpose | Evidence of fitness for one purpose: a release policy's rules decided on intervals. The framework and its governance are yours | Partly |
| 2.1 | An inventory of models in use | Each project's system versions, as the runs that evaluated them recorded them. Not an enterprise inventory | Partly |
| 2.2 | A model risk rating | Nothing | Not covered |
| 2.3 | Effort in proportion to risk | A policy per project: its bars, its confidence level, whether a passing release needs two people's approval | Partly |
| 3.1 | Lifecycle policies, documented | The release policy is a file, recorded by digest with every gate decided under it | Partly |
| 3.2 | Data suitable for its use | Suites versioned by content; whether they suit the use is yours | Partly |
| 3.3 | Development standards for performance and documentation | Rules with thresholds, and every run's record | Partly |
| 3.4 | Independent review of soundness and performance | A workspace gate that decides on evidence it verified itself, runs signed by a runner your platform team registers, and a decision record a reviewer checks offline. The review function is yours | Partly |
| 3.5 | Deployment with quality and change control | Nothing: a signed run attests what a runner executed, not what serves production | Not covered |
| 3.6 | Monitoring and decommission | Production review and freshness for monitoring; decommission is not covered | Partly |
| Approval | Approval before use, with residual risk affirmed | A passing release can wait for two named people's approval, recorded and chained (Gating) | Partly |
Where to go next
- Gating: the release policy, approvals and the decision record.
- Risk ceilings: what a ceiling decides and does not say.
- What works today: everything that is built, and what is not.