What is AI risk assessment?
By Identra · Updated
AI risk assessment identifies what could go wrong in a specific AI use case, evaluates the likelihood and impact of that harm, and decides whether the remaining risk is acceptable. It links the system's data, permissions, behavior and business purpose to tested controls and an accountable approval decision.
What should an AI risk assessment cover?
One use case, in its real configuration. A model drafting blog posts and an agent reading employee records in the HR system are two assessments, even on the same model and the same contract.
Write down the purpose, owner, affected people, expected outputs and every action the system can take. Then follow the data through prompts, uploads, retrieval, tool calls, outputs and storage, including connected SaaS and cloud services. Ask how disclosure, a wrong decision, unfair treatment, an outage or an unauthorized action would hurt someone.
The result is a decision. Proceed, add safeguards, narrow the scope or say no. That gives AI governance something concrete to sign.
What goes on an AI risk assessment checklist?
Vendor documentation describes the product. It doesn't show how your tenant, connectors and permissions are set up, so collect evidence from the deployment itself. Unknowns go on the record, with a call on whether they block approval.
Data
- Question
- What sensitive information can get in or out?
- Evidence
- Data flow, allowed data types, disclosure test results
Access
- Question
- Which identities and resources can it use?
- Evidence
- Granted permissions, resource scope, access tests
Autonomy
- Question
- What can happen with no human review?
- Evidence
- Tool list, approval rules, how to stop it
Reliability and fairness
- Question
- Who gets hurt by wrong or uneven results?
- Evidence
- Evaluations on representative cases, expert review
Vendor and account
- Question
- Which terms and admin controls apply?
- Evidence
- The contract you signed, retention settings, workspace owner
Operations
- Question
- Who notices problems and handles changes?
- Evidence
- Named owners, activity logs, incident procedure
How do permissions and autonomy change AI risk?
Permissions set what it can reach. Autonomy sets what it can do without asking.
Don't wave read access through. Microsoft 365 Copilot can surface any SharePoint file the user already has access to, so an overshared folder becomes an answer to someone's casual question. No write permission needed.
Apply least privilege per identity and per tool, and check whether the agent's access was scoped to the task or inherited from an admin account. Plant a hostile instruction in a test document and see whether prompt injection can steer the system toward disclosure.
Human approval only counts when it's specific. The reviewer sees the action, the affected records and the evidence, and the action can't change between the click and execution.
What does one look like in practice?
Say support proposes an agent that reads tickets, searches internal guidance, drafts replies and issues refunds. The appeal is faster case handling. The worries are leaking one customer's data to another, obeying hostile instructions buried in a ticket and refunding the wrong amount.
As proposed, the connector can search every customer and refund directly. The team narrows retrieval to the customer on the ticket, splits drafting from refunds and has a reviewer approve recipient and amount. The billing system checks authorization independently.
Before launch they test cross-customer searches, a ticket carrying planted instructions, a duplicate refund and a dependency that times out. Approval covers that configuration only. Inaccurate drafts stay on the record as an accepted risk, and adding a data source or dropping the refund review reopens the assessment.
How do you reduce risk before approval?
Tie each control to a specific failure. A written policy won't show that a connector respects access boundaries. A test with two accounts in different roles will.
AI vendor risk management covers training use, retention, deletion, subprocessors and breach notice for the plan you're buying. Confirm who administers the workspace, too. A work email on a ChatGPT login doesn't make it a company-managed workspace.
- Cut connectors, permissions and tools the task doesn't need.
- Run ordinary tasks, misleading inputs and failures against written acceptance criteria.
- Confirm you can stop a run, revoke access and reconstruct what happened.
- Give every approval condition an owner and a due date.
How do you record the decision and keep it current?
For each credible harm, note the impact and the evidence behind your likelihood call. Keep risk before controls apart from residual risk after them. Don't invent precision you don't have. One severe open issue stays visible even if every other row looks fine.
Record the approved scope, the tested controls, the evidence gaps and the person who accepted what's left. Keep the test results with the decision.
Reassess when data sources, permissions, tools, model behavior, vendor terms or the purpose change, or after an incident. That cycle is AI risk management.
How Identra thinks about it
Identra gives risk assessments evidence from real use: which AI apps people use with work or personal accounts, which connected apps can reach company data, and which agents, MCP servers and plugins run on endpoints. Teams can then apply account and data policies, review recorded agent runs and track the outcome of each response.
Go deeper: AI security, built on identity
Frequently asked questions
What is the difference between AI risk assessment and AI risk management?
Assessment identifies and evaluates risk for one defined use case. Management is everything after: putting controls in place, accepting or avoiding risk, monitoring and revisiting decisions.
Does an approved AI vendor make every use case safe?
No. Approval for drafting public content says nothing about confidential records or autonomous production changes. Data, account, permissions and configuration decide suitability.
Who should own an AI risk assessment?
A business owner accountable for the use case, with security, privacy, legal, procurement and domain experts weighing in. Name the person allowed to accept the residual risk.
Do low-risk AI tools need a full assessment?
They need a short one that shows why they're low risk. A tool limited to public drafting needs far less evidence than an agent touching sensitive records.
Is AI red teaming the same as AI risk assessment?
No. Red teaming probes how a system can fail or be abused. Its findings feed the assessment, which also weighs business purpose, affected people, vendor terms and who accepts the risk.