What is AI security?
By Identra · Updated
AI security is the practice of protecting AI systems and their use from unauthorized access, data exposure, manipulation and misuse. It covers the applications, models, data, identities and tools involved, from an employee's chatbot session to an agent acting on company systems.
What does AI security cover at a company?
Start with what people actually do. A sales rep pastes a pricing sheet into ChatGPT while signed in with a personal Gmail account instead of the company's ChatGPT Enterprise workspace. A developer runs Claude Code in a repo, and it reads the .env file to debug a failing build. Microsoft 365 Copilot answers a question with a SharePoint file that was shared with the whole tenant years ago. An agent in Google Workspace holds an OAuth grant that lets it send mail as a manager.
Each of those is in scope. So is the build side if you train or host models. That means training data, model files, retrieval indexes and the packages they pull in.
Buying AI from a provider moves some of the engineering work to them. Your accounts, your data and the permissions you grant stay yours. AI governance decides who owns each use and what is allowed. Security is where those decisions get enforced.
How is AI security different from traditional cybersecurity?
Most of the risk is familiar. Stolen credentials, vulnerable packages and over-broad permissions still do the damage. What's new is that the system reads natural language from sources nobody vetted, and some of that text can steer it.
A ticket, a web page or a code comment can carry instructions. The model may follow them. So reviews have to look at what the model reads and what it is allowed to do with the answer.
Data access
- Control you already have
- File and record permissions
- What changes with AI
- Check the requesting user's permissions at retrieval time
Untrusted input
- Control you already have
- Input validation
- What changes with AI
- Retrieved text can contain instructions and must stay data
Execution
- Control you already have
- Restricted commands and API scopes
- What changes with AI
- Model-written arguments need validation before they run
Change control
- Control you already have
- Code and config review
- What changes with AI
- Retest when the model, prompt, tools or sources change
How does an ordinary AI workflow become an incident?
Say a support team gives an agent one job. Summarize the ticket, look up the customer and draft a reply. An attacker files a ticket that tells the agent to export every customer record to an outside address. That's prompt injection.
What happens next comes down to the agent's real permissions. If its lookup tool only returns the customer on the ticket, there's little to steal. If it has a bulk export tool and can email anyone, the ticket just became a breach. The check that stops it has to live in the export API. Asking the model nicely in a system prompt won't do it.
Plenty of incidents have no attacker at all. Someone pastes a production AWS key into a personal chatbot. Copilot surfaces a salary spreadsheet because its sharing settings were already wrong. All of it counts as AI data leakage.
Where should a security team start?
Pick one real business task and trace it end to end. Which app, which account, which data sources, which actions it can take. Note where data leaves an approved workspace and where the workflow can change a system of record.
Look for shadow AI, but don't wave approved apps through. Approving Claude for the company says nothing about the personal Claude account someone uses on the same laptop. For agents, tie every credential to a named owner. AI agent identity covers how.
- Give every AI app and agent an owner and a stated purpose. Remove integrations nobody uses.
- Write down which data can go into which service, then apply it to prompts, uploads and tool calls alike.
- Scope agent credentials to the task with least privilege. Keep secrets out of prompts.
- Require human approval for actions that are hard to undo, and show the approver the real target and data.
- Test the full workflow with hostile documents and unexpected tool requests. The agentic AI security entry goes deeper on agents.
What should be ready before something goes wrong?
A written AI incident response runbook, before access expands. Responders need to know how to stop an agent mid-run, disconnect an integration, revoke a token and keep the records. Logs should link each request to the tool call and its result without copying every sensitive prompt.
After an incident, fix the permission that allowed the damage. Rewording the prompt leaves it in place. Rerun the original attack before you turn access back on.
How Identra thinks about it
Identra shows security teams how AI is used across the browser, the endpoint and connected identity, SaaS and cloud providers. Teams can allow, redirect or block supported browser AI apps by signed-in account, protect sensitive prompts before they are sent and restrict supported agent tool actions by policy. Analysts can revoke risky OAuth grants, and every response records its result.
Go deeper: AI security, built on identity
Frequently asked questions
Is AI security the same as using AI for cybersecurity?
No. AI security protects AI systems and how people use them. Using AI for cybersecurity means applying it to work like alert triage or malware analysis. Many teams do both.
How is AI security different from AI safety?
AI safety is about harmful model behavior in general. AI security deals with attackers, data exposure and misuse of access. They overlap when a manipulated model causes real harm.
Do we need AI security if we only use hosted chatbots?
Yes. People paste company data into hosted chatbots, and connected assistants can read mail and files. You still need rules for accounts, data, integrations and incidents.
Can prompt filtering prevent prompt injection?
It catches some attempts. It can't guarantee anything. Limit what the agent can reach and enforce authorization outside the model.
Who should own AI security?
Security coordinates with identity, endpoint, app and data teams. Each workflow also needs a business owner who approves its access and helps when something breaks.