How to Build a Pre-Deployment AI Risk Assessment

TL;DR

A pre-deployment AI risk assessment is a structured process that forces an organisation to answer the questions it would otherwise avoid: What exactly could go wrong here? Who would be harmed? How severely? And what would we do about it? This guide provides the specific method — inputs, outputs, and decision criteria.

When to Run This Assessment

Every AI system deployment, regardless of scale or perceived risk, warrants some version of this assessment. The depth of the assessment should be proportional to the stakes — but skipping it entirely is never the right call.

Run a full assessment when:

• The system makes or influences decisions about people (hiring, lending, healthcare, education)
• The system generates content that will be published without human review
• The system handles sensitive personal data
• The system will be used by a large number of people

Run an abbreviated assessment when:

• The system is an internal productivity tool with no external-facing outputs
• The system’s outputs are always reviewed by a human before action is taken
• The system is a well-understood tool in a stable, low-stakes domain

The Assessment Process

01 / Define the System

Before assessing risk, the system must be precisely defined. Vague definitions produce useless risk assessments.

Document:

• Exactly what decision or output does this system produce?
• What data does it consume as inputs?
• Who operates it, and who receives its outputs?
• What actions are taken based on those outputs?
• Are those actions reversible?

The irreversibility question is the most important. An AI system that flags content for human review is fundamentally lower risk than one that automatically removes content. The distinction should shape every other decision in the assessment.

02 / Identify Affected Populations

For every group of people that the system’s outputs could affect, characterise:

• The nature of the potential effect (beneficial, neutral, harmful)
• The severity if harm occurs (minor inconvenience to life-altering)
• The reversibility of the harm
• Whether the group has meaningful recourse if harm occurs

Pay specific attention to groups that are underrepresented in your training data — they are statistically most likely to receive worse outcomes from your system.

03 / Enumerate Failure Modes

For each component of the system, identify the most plausible failure modes:

Be honest about severity and probability. The purpose of this exercise is not documentation — it is to surface the risks that require mitigation before deployment.

04 / Assign Risk Scores and Thresholds

For each identified failure mode, assign a composite risk score:

Risk = Severity × Probability × (1 − Reversibility)

This formula forces the assessment to weight irreversible harms more heavily — consistent with good governance practice.

Establish thresholds before scoring:

• Green (deploy): Risk score below threshold, standard monitoring sufficient
• Yellow (deploy with controls): Risk score elevated, additional mitigations required
• Red (do not deploy): Risk score above threshold, deployment not approved until mitigated

05 / Design Mitigations

For each yellow or red risk, define specific mitigations:

• Technical: system prompt constraints, output filters, human review gates
• Operational: training for operators, clear escalation paths
• Monitoring: specific metrics to track, alert thresholds, review cadence

A mitigation is only valid if it meaningfully reduces severity, probability, or irreversibility. “We will monitor it” is not a mitigation for a high-severity risk.

06 / Document and Sign Off

The completed assessment should be:

• Reviewed by someone who did not conduct it
• Approved by the accountable AI Risk Owner
• Stored with version control so future changes can be tracked
• Referenced in the system’s ongoing monitoring plan

The assessment is not filed and forgotten. It is the baseline against which future incidents and monitoring results are compared.

Common Assessment Failures

Scope creep avoidance: Teams tend to assess the system they intended to build, not the one they actually built. Assess what the system does, not what it was designed to do.

Optimism bias: Risk assessors who want the project to succeed underestimate probabilities. Bring in an external reviewer for any high-stakes system.

Missing the indirect effects: The most consequential harms from AI systems are often indirect — effects on people who never interact with the system directly. Mapping affected populations carefully is the only way to catch these.

Supplier Evidence

Added 3 August 2026.

Most of the assessment above examines a system you can inspect. The part you cannot inspect is the supplier’s, and since 2 August 2026 the consequences of leaving that gap undocumented have changed.

The EU AI Act’s enforcement routes became operational on that date, including a channel for downstream providers to escalate alleged breaches by general-purpose AI model providers. Separately, Article 25(4) of the Act — as amended by the Digital Omnibus in July 2026 — now requires that the provider of a high-risk system and any third party supplying an AI system, AI model, tools, services, components or processes specify by written agreement the “necessary information, capabilities, technical access and other assistance” needed for compliance. “AI model” is newly added; model suppliers are explicitly in scope.

Those obligations bite from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I products, so there is time. The reason to start now is that renegotiating a contract takes longer than writing an assessment.

Add a supplier-evidence step to the assessment, and record four things:

• What you asked for. Model documentation, evaluation results, known limitations and failure modes, technical access for testing and validation, incident-notification commitments.
• What you received. Including the date, and whether it was specific to your use case or generic marketing material.
• What is missing, and what it prevents. An unanswered request is evidence — but only if you kept it and recorded the operational consequence. “The supplier did not provide bias testing for our population, so we conducted our own” is a defensible position. Silence is not.
• What you decided. Whether the gap was accepted, mitigated, or blocked deployment, and who made that call.

Two cautions on supplier assurances:

• A signed voluntary code is not evidence of operation. Roughly 190 organisations signed the Commission’s Code of Practice on Transparency of AI-generated Content in July 2026. Signing is not certification, not an audit, and not Commission approval. If anything it gives you something specific to hold a supplier to — not something to rely on.
• A vendor’s safety record describes a configuration. July 2026 disclosures from two major labs showed models behaving very differently when production classifiers and monitoring were absent. If your deployment sits upstream of, disables, or bypasses those controls, the vendor’s evidence does not cover you. The Claude evaluation boundary failures case study sets out how sharply that boundary can move.

Supplier assistance does not transfer the duty. It remains your assessment, your oversight and your evidence.

Jurisdiction-Specific Fairness and Outcome Evidence

Added 10 August 2026.

On 7 August 2026 the US Federal Trade Commission announced it will no longer pursue disparate-impact claims under any of its authorities, nor antidiscrimination claims under Section 5 of the FTC Act. It will continue appropriate disparate-treatment enforcement under the Equal Credit Opportunity Act.

Two details that get lost in summaries:

• The FTC did not merely deprioritise these theories — it asserts it never had the authority, describing its previous disparate-impact claims under both Section 5 and ECOA as ultra vires. That is a stronger and more durable position than an enforcement-priority shift.
• ECOA is not untouched. Disparate-impact claims under ECOA were abandoned too. What continues there is disparate-treatment enforcement.
• The policy statement is non-binding. Footnote 1 states it creates no rights and binds neither the FTC nor the public.

What this changes in an assessment: the regulator map, not the measurement.

It pre-empts no federal, state or local law, binds no court, amends no statute, and leaves deception, unfairness and other statutory theories intact. State attorneys general, sector regulators and private litigants are unaffected. An organisation deploying a credit or decision system in the US has a narrower FTC exposure and the same everything-else.

So keep measuring group outcomes. Measurement surfaces drift, data defects, pretext and exposure under regimes the FTC does not administer — and a statistically neutral system is not thereby free of intentional discrimination. Note the countervailing constraint honestly: collecting and using protected-class data must itself be lawful, and the lawful basis for doing it belongs in the assessment alongside the testing plan.

The wider point for anyone assessing a system deployed across jurisdictions: maintain a regulator-scope register, because a single regulator’s retreat is not a change in the underlying obligation. In the EU, over the same period, the direction has been the opposite.

Reassessment After a Capability-Threshold Change

Added 10 August 2026.

Most assessment processes assume the system under assessment holds still. It does not. Two things move underneath you: the vendor’s model, and the vendor’s own view of what that model can do.

On 7 August 2026 OpenAI announced it could not rule out that its forthcoming Astra model meets the Critical cybersecurity threshold in its Preparedness Framework, and imposed additional internal controls in response. Whatever happens to Astra, the mechanism is the point: a capability finding triggered a restriction automatically, before any incident.

Build the deployer-side equivalent. Decide, at assessment time, the answers to:

• What evidence would make us restrict or stop this deployment? Name it now, while nobody is under pressure.
• Who is authorised to pull that brake, and does it require anyone’s agreement?
• What is the maximum time between a vendor capability disclosure and our reassessment?
• What do we do in the interval — carry on, restrict scope, or suspend?

Two cautions on borrowing a vendor’s framework wholesale. First, a vendor’s threshold is not your threshold — theirs concerns what the model can do, yours concerns what it can do with your permissions, your data and your credentials. Second, a published framework is not necessarily a followed one: OpenAI’s own framework prescribes “halt further development” at Critical, while what was announced was a pause of non-conforming activity. Read what the vendor did, not only what its policy says.

Retesting After Vendor Control Changes

Added 10 August 2026.

A safeguard change can invalidate your deployment evidence while every identifier you track stays exactly the same — same product, same model name, same API, same integration.

On 7 August 2026 Anthropic retrained Fable 5’s biology classifier so more benign health, educational and clinical queries are answered directly rather than falling back to another model. In its own testing, biology-related fallbacks fell by about 85%. Nothing you would notice in a dependency file changed at all.

The retest, when a vendor announces a safeguard or routing change:

• Routing regression tests. Representative and adversarial requests across the affected categories. Which model answers now — and is that different from your baseline?
• Review-threshold revalidation. If content previously reached a human because it was refused or rerouted, and no longer is, your escalation trigger has moved without anyone deciding to move it.
• Output accuracy in the newly permitted range. More answers in a domain where wrongness is invisible to a non-specialist is precisely where a validation layer earns its cost.
• A vendor-change register, with a named owner who watches announcements and holds authority to trigger retesting, rollback or suspension.

And read the announcement critically. Anthropic’s 85% is self-reported internal testing, measured against a baseline it had deliberately set at maximum restriction at launch, and it concedes false positives inevitably remain without quantifying them. All three qualifiers are on the vendor’s own page — none of them appear in a headline.

Testing Cumulative Agent Behaviour

Added 10 August 2026.

Assessment usually asks whether each action an agent can take is acceptable. For agents, that is the wrong unit. Individually permitted actions combine into transactions nobody authorised. Three purchases each under the approval limit. A transfer that uses an account looked up two steps earlier. A sequence of reads that assembles something no single read exposed.

AWS shipped a partial answer on 6 August 2026: temporal policies in Amazon Bedrock AgentCore, which evaluate an agent’s session history before permitting a call, alongside gateway rate limits. Useful, and worth understanding precisely:

• Limits scope on several dimensions — target name, tool name, model ID and identity claims among them. Identity is the headline case, not the only one.
• Token rate limits apply to inference targets only. Requests and connection rate are broader.
• Connection limits are a rate — connections per second — not a ceiling on simultaneous sessions.
• Unmatched callers can fall through to service quotas unless a catch-all is configured. AWS’s own guidance warns about this.

What no platform decides for you: which business invariants matter, what the cumulative threshold should be, who approves an exception, and what happens on denial. A temporal policy enforces a rule consistently — including a wrong one.

Assess it by asking: what is the worst sequence of individually-permitted actions this agent could perform? Then check whether anything would stop it.

Reusable Agent Skills: Ownership, Evaluation and Withdrawal

Added 10 August 2026.

A governed model and a secured tool can still be connected by an unowned, outdated or over-permissioned reusable instruction. Skills, prompt packs and agent templates are shared like code and governed like documentation.

In a Google Cloud engineering account published 3 August 2026, the team described how it manages its own skills repository: structural, link and guardrail checks, evaluation prompts with scoring rubrics, repeated runs, and weekly quality checks running scheduled evaluation jobs across the full library to catch regressions.

Two things to hold in view. This is a practice account by one engineering team, not a standard, policy or certification — it is written in the first person and describes internal working practice. And its scope is Google’s own repository and contributions from Google product teams; nothing in it governs third-party or community-authored skills. Notably, exported public versions strip three things: internal assets, ownership information, and evaluation suites. What an external deployer can independently verify is therefore limited.

Your controls, for any shared skill you rely on:

• A named owner per skill, not a repository.
• Permission mapping — what tools and data does this instruction reach?
• Versioning and an approval baseline, so you can tell what changed.
• Recurring evaluation, because models, APIs, source material and agent frameworks all move after approval.
• A withdrawal route. Deciding to stop using a skill is a control only if someone can actually remove it.

Regulatory-Sandbox Readiness for Legal AI

Added 10 August 2026.

On 3 August 2026 the UK opened applications to the Legal Services AI Growth Lab, a coordinated engagement programme run with the Ministry of Justice and the Department for Business, Innovation, Science and Trade, alongside the Legal Services Board, Solicitors Regulation Authority, Council for Licensed Conveyancers and the Information Commissioner’s Office. Applications close at 11:59pm on 27 September 2026; participation lasts up to nine months.

Read the eligibility bar accurately, because it is lower than it sounds and differently aimed. The programme asks applicants to show readiness to engage with regulators at pace — a sufficiently developed product, partnerships in place where needed, and the organisational capacity and commitment to engage throughout. It states plainly: “We’re not expecting products to be market-ready.” Evidence strengthens an application rather than qualifying it.

Nor does every participant get all six bodies. Engagement is with relevant regulators, and which ones depends on the proposal.

What it does not do, and this is the part worth putting in front of anyone excited about it:

• No approval, endorsement, authorisation, exemption or safe harbour. The guidance says so directly.
• No change to the law. Questions must be answerable within existing regulatory frameworks.
• No transfer of professional responsibility. Participants remain responsible for complying with everything that already applies to them.

Which makes it a useful lens for assessment even if you never apply. The application asks what a good pre-deployment assessment asks: what exactly is the product, what specific regulatory question does it raise, who owns the answer, and can you evidence any of it? Regulatory engagement cannot substitute for an undefined use case, an absent control owner, or an inability to show the product is ready to be examined.

Sources