Sponsored by Box

Why Identity and Access Controls Are Not Enough to Secure Autonomous AI Agents

Identity and permissions remain essential to enterprise security, but they are no longer enough to protect autonomous AI agents. Traditional access controls define what an agent can reach—not necessarily what it will do once it begins operating independently. In seconds, an AI agent can turn legitimate access to corporate data into unintended or harmful behavior.

Heather Ceylan, chief information security officer at Box, says this emerging risk is pushing enterprises to expand their approach to AI security. Instead of focusing only on access management, organizations must also control how AI agents execute tasks, interact with tools, and use business content.

“Access control and permissions are fundamental, but the challenge is that they are designed for humans,” Ceylan says. “Permissions remain the foundation, but we also need to consider how agents scope their permissions.”

Traditional access control was built for a slower and more permissive environment. A human employee may have persistent access to folders created 10 years ago without ever opening them—or even remembering that the access exists. An AI agent, however, can examine those permissions at machine speed, quickly discovering forgotten access paths, outdated permissions, and security misconfigurations.

Identity-based controls and carefully scoped permissions remain a critical first line of defense. However, as organizations deploy more autonomous AI agents, clean access hygiene and continuous monitoring become increasingly important.

A growing number of incidents demonstrate the potential impact. In one example, a model escaped the sandbox it was supposed to remain within. In other cases, systems reached data or services they were not authorized to access. While the circumstances differ, the underlying risk is similar: an AI agent identifies an available path and follows it.

Access control alone cannot protect autonomous AI agents

Security challenges become more complex when AI agents receive broad, persistent permissions designed to support many workflows. For example, an agent may legitimately need to use dozens of tools and perform numerous actions while reading and writing files across multiple departments.

Granting all of those permissions at once, however, increases the potential blast radius of a mistake. A stronger security model gives an agent access only when it is required and limits that access to the specific task being performed, Ceylan says.

“You need permissions that change based on what the agent is being asked to do and when they need to perform that action,” she explains. “If you have one step and only need two tools, you should limit your scope to just those two. As you narrow down the permissions to the task at hand for your agents, the number of chances that a particular step will be performed incorrectly decreases accordingly.”

AI agent security requires control over execution

Effective AI agent security is not only about whether an agent can access specific data. It is also about whether the agent should perform a particular action at a particular stage of a workflow.

This distinction separates permanent permissions from task-based or just-in-time permissions. An agent may have a legitimate reason to access a finance folder, for example, but it may not need to copy thousands of files to a new location—even if a user or another system instructs it to do so.

“Employees with access to payroll data could be directed to retrieve payroll records and write them to a publicly shared folder, potentially exposing compensation information for the entire company at once,” Ceylan says. “Although all access checks passed, this behavior still has catastrophic consequences.”

Prompts alone cannot reliably control autonomous AI agents. Instructions can change, an agent can absorb malicious directions, or a file it reads can contain content designed to manipulate its behavior. Persistent safeguards must exist below the prompt layer, including controls around tool calls, data access, and the actions an agent is permitted to execute.

By defining acceptable actions in advance, organizations can keep agents within a controlled operating range—even when prompts or data inputs are manipulated.

Why traditional content platforms are not designed for AI agents

Most enterprise AI agents interact with unstructured content, including contracts, policies, customer records, reports, and other business files. This content represents a significant portion of corporate data, yet much of it remains stored in systems designed around human filing practices, including network drives, legacy enterprise content management platforms, and disconnected SaaS applications.

These systems were generally built to answer a simple question: Does this person have permission? Their answers often rely on folder-level access controls that may not have been reviewed or audited for years.

Traditional content platforms also frequently lack the metadata, classifications, and detailed activity records that AI agents need. Adding an AI connector to an existing system does not automatically resolve those limitations. Instead, it may give an AI agent the same visibility gaps—only at machine speed.

“Every action an agent takes ultimately resolves to content,” says Ceylan. “If your content layer can’t tell you what it holds, who it belongs to, and what should never leave the content layer, there’s nothing you can control.”

How to determine which AI actions need human approval

Historically, many organizations assumed that secure AI automation required a human to approve every action. As enterprises gain experience with autonomous AI agents, that approach is evolving. Box categorizes agent activities into three broad levels: fully autonomous actions, supervised actions, and high-risk actions that require human approval.

• Fully autonomous actions: These are appropriate for tasks that are reversible, limited in scope, logged, free from untrusted inputs, and unlikely to cause significant harm if performed incorrectly.

• Actions requiring monitoring: These tasks can proceed with appropriate alerts, activity tracking, and rollback capabilities. Organizations can use monitoring to identify and reverse problems as trust in the agent develops.

• Irreversible and high-risk actions: These actions should be routed through a human reviewer. Examples include deleting large numbers of files, modifying critical records, or removing a primary folder structure.

Every organization should define the boundaries between these categories according to its risk tolerance, regulatory obligations, and business requirements.

Box’s approach is to place controls directly in the platform rather than relying entirely on individual workflows. Protections such as data classification, labels, retention rules, and expiration dates can then be enforced consistently without requiring a human checkpoint for every low-risk action.

“Instead of blocking actions at the end, you should apply the correct configuration at the beginning,” Ceylan says.

Building long-term trust in autonomous AI agents

Organizations can approve access at a single point in time, but trust in an AI agent must be earned through ongoing observation. Security teams need to understand how agents behave, how they interact with one another, and how they use or build on the output of other agents.

Public AI security frameworks can be useful, but they may lag behind rapidly changing technology or remain too abstract for day-to-day implementation. Box instead focuses on a smaller set of operational principles, including tightly scoped agent identities, clearly defined actions, reliable rollback procedures, tiered approval requirements, and rapid testing and iteration.

“The sanctioned path needs to be the fast path, because if teams aren’t given a safe way to experiment, they tend to bypass controls altogether,” Ceylan says. “The security leader’s job is to provide a way to move quickly without going outside the guardrails.”

Maintaining trust also depends on visibility into how an agent actually operates. Traditional monitoring tools were designed primarily for human users, so their behavioral baselines may not accurately identify suspicious AI agent activity.

Useful agent monitoring begins with comprehensive logging, but many AI agents start as experiments and may not be connected to existing logging infrastructure. In addition, the most important warning signs may involve sequences of actions rather than a single event. For example, one agent’s output may become another agent’s input, creating a chain of activity that is difficult to detect with conventional tools.

As organizations deploy more autonomous AI agents, they need visibility into not only what those agents are allowed to access, but also what they actually do. That visibility should exist at the content layer, where data is stored, classified, governed, and ultimately used.

Learn more about AI agent security and governance.

Sponsored articles are content created by companies that pay us to post or have a business relationship with VentureBeat, and are always clearly marked. For more information please contact us [email protected].