What is an AI security program?
By Identra · Updated
An AI security program is the ongoing work of protecting the data, identities, systems and actions involved in an organization's AI use. It ties named owners, an AI inventory, enforceable policies, tested controls and incident response together across purchased AI services and internally built systems.
What does an AI security program cover?
Employees in ChatGPT, Claude and Gemini. Agents acting on company systems. AI features that appeared inside SaaS you already pay for, like Microsoft 365 Copilot. And whatever your engineers build on a model API.
Scope follows the workflow: who starts it, what data it gets, which tools it can call, what it can change.
AI governance sets decision rights. The program turns those decisions into assigned work and evidence. A policy PDF on the intranet won't tell you whether revoking an agent's token actually works.
Who owns it?
One person, with enough authority to pull in security, IT, engineering, privacy, legal and the business. Shared ownership with nobody named stalls the first time two teams disagree.
Every material AI workflow gets an owner too. That person can explain its access, approve changes and retire it when the need goes away. Write down who grants exceptions and who can pull the plug.
How do you build the AI inventory?
No single source has it. Pull procurement records, Okta or Entra ID sign-in logs, OAuth grants in Google Workspace and Microsoft 365, browser and endpoint inventories and cloud accounts. Then go talk to the teams. Shadow AI includes personal accounts, local coding agents and AI features someone switched on inside an app you approved years ago.
For each entry, record purpose, owner, account, reachable data, permissions, tools and approval status. Extensions, plugins, skills and MCP servers belong on the list. For in-house systems, add the models, data sources and dependencies.
Rank by plausible harm. A summarizer for public blog drafts and an agent with write access to Terraform state don't share a queue. An AI risk assessment handles the detail. Where you can't tell what something can reach, write that down as a finding and give it an owner.
Which controls should the program require?
Start with an AI acceptable use policy: approved apps and accounts, allowed data, allowed agent actions, when a human signs off. Each rule then needs a control, someone who runs it and a test. Least privilege applies to people, agents, tools and service accounts alike.
- Give people an approved route for real work. Blocking without one pushes them to personal accounts.
- Separate read access from the authority to change things.
- Test prohibited actions with synthetic data, including ones triggered by a poisoned document.
- Every exception gets an owner, a scope and an expiry date.
Browser
- What to control
- Approved accounts, prompts, uploads, extensions
- One way to verify
- Try a prohibited upload with synthetic sensitive data
Endpoint
- What to control
- Local AI apps, agent tools, file access, shell commands
- One way to verify
- Confirm an agent can't modify a protected test file
Identity, SaaS and cloud
- What to control
- OAuth grants, service identities, resource permissions
- One way to verify
- Remove a test grant and confirm the access fails
What does a pilot look like?
Say support pilots an agent that searches tickets and drafts replies. The integration it shipped with can also delete tickets. Meanwhile, engineers paste troubleshooting logs full of credentials into personal AI accounts whenever the agent can't answer.
The program owner gets support, IT and security into one review. Deletion permission goes. Retrieval narrows to the customer on the ticket. Replies stay as drafts until a person sends them. Engineers get an approved place to analyze logs.
A test ticket carries hidden instructions to export customer records. The export fails under the agent's permissions. The team also rehearses disabling the integration and revoking its token, and the pilot doesn't expand until both work.
How do you know the program is working?
Look at exposures closed. Permissions confirmed removed. A drill that actually stopped the agent. Exceptions that expired and got cleaned up. Alert volume says little, since a quiet dashboard can mean less activity or less visibility.
Run an AI incident response drill. Who stops the agent, who revokes access, who preserves evidence, who decides on recovery. Confirm containment in the underlying service. Closing a chat window can leave a token valid and a background job running.
Prompts and outputs in investigation records often contain sensitive data, so protect them. Retest when tools, permissions, models or data sources change.
How Identra thinks about it
Identra shows security teams AI use across the browser, macOS and Windows endpoints, and connected identity, SaaS and cloud providers. Teams can set account-aware browser policies, protect sensitive prompts, check endpoint agent tool calls against policy and have an analyst revoke risky OAuth grants, with the result of every response recorded.
Go deeper: AI security, built on identity
Frequently asked questions
How is an AI security program different from AI governance?
Governance sets accountability and expectations across security, privacy, legal and business concerns. The security program runs the controls, testing, monitoring and response that protect AI workflows.
Do we need a dedicated AI security team?
You need an accountable owner and people with time to do the work. Existing security, IT, identity and engineering teams can carry it if responsibilities and escalation are clear.
Does this apply if we only buy AI software?
Yes. Bought AI services receive company data and act through company permissions. The program still covers configuration, employee use, integrations, vendor review and response.
Should we begin by blocking all unapproved AI?
Shut down known dangerous exposures right away. For the rest, use discovery and workflow reviews to decide what to enforce, and offer an approved option so people can still do their jobs.
What changes if we build our own AI systems?
The program grows to cover model and data provenance, development access, dependency review, application testing and deployment controls. Changes to models, retrieval sources, prompts and tools need security checks.