Satya Nadella Is Right: We Need to Treat AI Models as Insider Risks

Satya Nadella just published one of the most important perspectives I've read on securing AI in the enterprise. In his essay, Models as Insider Risks in the Super Intelligence Era, he argues that as we deploy increasingly capable AI systems across our organizations, we need to fundamentally reconsider how we establish trust.

His central argument is simple:

“We need to separate the supply of intelligence from the authority over it.”

I couldn't agree more.

For years, we've been building increasingly sophisticated security architectures around a fundamental assumption: any actor with access to sensitive enterprise systems can introduce risk, regardless of whether that actor is malicious, negligent, or compromised.

Now we're introducing an entirely new class of actors into those same environments. We're granting AI models access to sensitive information, connecting agents to enterprise applications, and allowing them to execute increasingly consequential actions on our behalf.

And we're doing it with systems whose internal decision-making processes we cannot reliably reconstruct.

Satya's argument is that the solution isn't to wait for models to become perfectly aligned, predictable, or explainable. It's to apply the security principles we've spent decades developing for powerful actors operating inside our organizations.

Treat AI models as insider risks.

This is a remarkably important framing, and I think its implications extend well beyond AI safety. It provides a foundation for how enterprise security itself needs to evolve.

We've spent decades learning not to blindly trust insiders

What makes Satya's argument particularly compelling is that the security problem he's describing is fundamentally familiar.

Every enterprise already operates with thousands of trusted actors. Employees, contractors, partners, service accounts, and privileged administrators all receive some level of authorized access to information and systems.

We've learned, often through painful experience, that authorization alone tells us very little about whether an action is appropriate.

An employee might have legitimate access to customer information and still misuse it. A privileged administrator can make an honest mistake with catastrophic consequences. An external attacker can compromise an authorized identity and operate entirely within its existing permissions.

This is why we've developed identity management, least privilege, access governance, activity monitoring, behavioral analytics, independent auditing, and incident response.

Collectively, these mechanisms allow organizations to grant authority without requiring unconditional trust in the actor exercising it.

Satya is proposing that we apply the same philosophy to intelligent systems.

An AI model doesn't need to have malicious intentions to create risk. It can misunderstand an instruction, encounter adversarial content, make an incorrect judgment, or be compromised through the environment in which it operates.

And unlike traditional software, where we can often trace an outcome to a specific code path, the behavior of a sufficiently complex AI model cannot always be explained through a deterministic execution sequence.

Yet we're giving these systems access to some of our most valuable business assets.

The consequences of this mismatch between capability, authority, and accountability are becoming increasingly significant.

What makes the insider-risk analogy so powerful is that it doesn't require us to solve the philosophical questions about model intent or consciousness. We already have a practical security discipline for managing authorized actors whose behavior can be unpredictable, inappropriate, or harmful.

We can apply it today.

The model cannot be responsible for policing itself

One of the most important parts of Satya's essay is his insistence that the controls governing models must exist outside the models themselves.

He draws on a longstanding information security principle: a program should not be able to bypass or tamper with the mechanisms enforcing its permissions.

That principle has been central to trusted computing for decades, and it's particularly relevant to agentic AI.

Think about how an AI agent operates.

There's the underlying model, responsible for interpreting information and generating decisions. There's the harness or orchestration environment that manages execution. And there's the action space: the tools, systems, data, and operations the agent can access.

These are distinct layers, even when a vendor packages them together.

A model may decide that retrieving an entire customer database would help accomplish a task. Whether that action is permitted must be determined by controls outside the model's reasoning process.

The same is true for sending an email, deploying code, modifying infrastructure, or moving sensitive data between applications.

A system prompt telling the model not to perform an unauthorized action is useful guidance. It cannot be the ultimate enforcement boundary.

The enforcement mechanism needs to be independent, and organizations must retain the authority to define, inspect, and change those boundaries.

This distinction becomes even more important as models gain autonomy.

When an AI assistant only generates text, we can often evaluate its output before taking action. When an agent independently executes a chain of operations across multiple systems, the enterprise is delegating real authority.

And that authority must remain governable independently of the intelligence using it.

Observability is the foundation of trust

Satya also makes an important distinction between understanding a model's internal reasoning and observing what it actually does.

He argues for chain-of-thought transparency, while acknowledging that model-generated reasoning traces are not necessarily complete or faithful explanations of behavior.

This creates an uncomfortable situation for organizations relying on models to explain their own actions.

An agent might provide a perfectly plausible explanation for why it accessed a sensitive file. That explanation doesn't independently establish what triggered the access, whether the action was authorized, or whether information was subsequently transferred elsewhere.

Having another model evaluate that explanation may improve our understanding, but it introduces a second model whose judgments can also be imperfect.

Satya describes the danger of building nested black boxes: opaque models operating inside opaque orchestration environments, supervised by other opaque models.

The solution is independent observability.

Every meaningful action needs to produce reliable evidence that exists outside the model's own control.

That includes which identity initiated the action, what task was being performed, which instructions and data influenced execution, what tools were invoked, what resources were accessed, and what changes occurred.

But enterprise observability also needs to extend beyond an individual agent runtime.

Consider an agent that receives a task from an employee, retrieves customer records from Salesforce, accesses a document in Google Drive, and sends information through Outlook.

The agent's execution trace is one part of the picture. Salesforce, Google Drive, Outlook, the identity provider, and the employee's surrounding activity can provide additional independent evidence.

And that evidence matters, because the agent's own account of its actions should never be the only record available to investigators.

This is where I think the connection to insider risk becomes particularly important.

Security teams have long understood that individual actions can appear entirely legitimate in isolation. The meaning of those actions often emerges only after reconstructing their sequence and relationships.

A document download is not necessarily risky. Neither is an external email. Neither is an unusual database query.

But when those actions are connected to the same identity, business process, sensitive information, and destination, an entirely different picture may emerge.

The same principles must now apply to synthetic actors.

We need to understand not only what an AI system was technically allowed to do, but what it actually did, why the activity mattered, and whether it remained consistent with the business purpose for which authority was granted.

Satya's seven principles should become enterprise security requirements

What I particularly appreciate about Satya's essay is that he doesn't stop at identifying the problem. He proposes seven concrete principles for designing trustworthy AI systems.

They deserve serious attention from anyone building or deploying AI in the enterprise.

1. Model diversity. No single model should become the sole authority over critical outcomes or the only mechanism responsible for validating its own work. Independent verification and diversity of approaches can reduce common failure modes, although they don't eliminate them entirely.
2. Observe everything. Every meaningful model action must leave reliable, human-readable evidence. Enterprises need to reconstruct what happened using observable facts rather than relying exclusively on what a model claims to have done.
3. Verifiability. AI systems need continuous testing against failures, attacks, edge cases, and changing environments. Successfully completing a task is only one dimension of whether an agent is operating safely.
4. Independent controls. Organizations must determine what models can access and what actions they can perform. Those decisions must be enforced independently of model-generated instructions and outputs.
5. Independent auditability. The intelligence being evaluated cannot have exclusive control over the evidence used to evaluate it. Organizations need independent records, validation mechanisms, and oversight.
6. Containment. We should design for the possibility that an agent will behave unexpectedly or become compromised. That means bounded permissions, constrained execution, revocable authority, and mechanisms to interrupt harmful activity.
7. Incident disclosure. When systems fail, organizations need to understand the failure, determine which controls were insufficient, and share appropriate lessons so the broader industry can improve.

These are strikingly similar to principles that mature insider-risk programs have spent years implementing for human actors.

We establish identities. We limit privileges. We monitor authorized activity. We investigate deviations. We preserve evidence. We respond to incidents. We learn from failures.

The architecture Satya describes is an extension of that security discipline to a new class of actors.

And importantly, these principles must work together.

Comprehensive logging without investigation can create an enormous collection of unexplained events. Strong permissions without runtime visibility can leave organizations unaware of harmful behavior within authorized boundaries. Detection without containment can identify incidents that the organization is unable to interrupt.

A trustworthy system requires all of these capabilities operating as an integrated security architecture.

AI agents make the insider-risk problem exponentially more complex

There's another implication of Satya's thesis worth exploring.

Traditional insider-risk programs were built around relatively understandable relationships between people, identities, applications, and information.

AI agents introduce new layers of delegation and execution.

An employee may delegate a task to an agent. That agent may invoke additional agents, use multiple applications, execute code, operate across endpoint and cloud environments, and interact with data the employee never directly examined.

Some agents operate locally. Others execute remotely, inside cloud infrastructure or third-party services. Their activities may be distributed across systems that have no shared understanding of the original task.

The question of who actually performed an action becomes considerably more complicated.

Was a file accessed directly by an employee or by an agent operating under that employee's credentials? Did the employee explicitly authorize the transfer of information, or did the agent infer that it was necessary? Was an external instruction introduced through a compromised document or tool response?

An ordinary identity log may show authorized access while missing the crucial distinction between human and agent activity.

Even comprehensive agent tracing may fail to explain the organizational significance of what happened.

This is why securing agents exclusively at the model, prompt, or endpoint layer will be insufficient.

The enterprise needs visibility into how authority is delegated, how actions propagate between systems, and how agent behavior relates to business objectives.

An agent accessing a particular customer record might be acting appropriately when preparing a renewal. The same access could become concerning when the agent retrieves unrelated accounts, aggregates confidential pricing information, and transfers it to an unapproved destination.

Determining the difference requires understanding the employee, the agent, the task, the business relationships, and the information involved.

That's the investigative challenge enterprises are about to face at scale.

The connection to Above's founding thesis

Satya's essay resonates deeply with why we started Above Security.

Our original thesis was that insider risk had become too complex to manage through static policies, isolated signals, and manual investigations.

Traditional tools were reasonably good at determining whether an event violated a predefined rule. Understanding whether a sequence of authorized actions represented meaningful business risk was much harder.

We believed AI could fundamentally change that equation by enabling continuous behavioral investigation across the enterprise.

That approach is now becoming relevant to an entirely new population of insiders.

As AI agents receive access to enterprise identities, applications, endpoints, and sensitive information, the same investigative principles apply. We need to correlate their actions across systems, understand the context in which those actions occurred, reconstruct timelines, and provide evidence that security teams can independently evaluate.

At Above, we're building AI investigative agents to help security teams perform that work continuously, connecting technical activity with the organizational context that gives it meaning.

This is also part of the thinking behind the Synthetic Insider Threat Matrix, which we introduced with Forscie through Above Theory.

The framework extends established insider-threat thinking to synthetic actors, creating a common language for understanding how their directives, configurations, invocations, and visibility limitations can contribute to adverse outcomes.

The goal is to make these risks understandable, investigable, and ultimately manageable.

I see this as one part of the architecture Satya describes. Independent behavioral investigation complements the external controls responsible for restricting and containing agent activity.

And there's an important responsibility on our side, too.

If AI is investigating AI, those investigations must be grounded in independently collected evidence. The investigative model cannot become the sole authority over the facts. Its conclusions need to be explainable, reviewable, and subject to human oversight.

The same principles we advocate for securing enterprise AI must apply to the AI systems we build for security.

The next generation of enterprise security will be built around accountable authority

For decades, we've designed enterprise security around controlling access to systems and information.

We're now entering a period in which intelligent systems will increasingly exercise that access on our behalf.

As their autonomy increases, the question of trust becomes unavoidable.

Can we establish who or what initiated an action? Can we reconstruct what happened across multiple systems? Can we independently verify that the action was appropriate? Can we recognize when authorized behavior creates risk? And can we intervene before that risk becomes an incident?

These questions will become foundational to enterprise AI deployment.

Satya's argument is particularly important because it gives us a practical direction. We don't need to solve every question about model alignment or internal reasoning before implementing meaningful safeguards.

We already know how to establish independent authority, observe trusted actors, investigate behavior, preserve evidence, and contain harmful activity.

Now we need to adapt those capabilities to models and agents operating inside the enterprise.

I expect this will become one of the defining changes in cybersecurity over the coming years. Insider risk will increasingly encompass both human and synthetic actors, while AI governance will need to move beyond policy documents and model evaluations into continuous operational oversight.

The most valuable AI systems will be those that organizations can give meaningful authority to while retaining independent control and accountability.

Satya has articulated the trust architecture for that future remarkably well.

We've spent decades learning how to secure trusted human access. It's time to apply that same rigor to the intelligence we're bringing into our organizations.