NVIDIA builds AI agent safety layer to prevent autonomous drift
NVIDIA introduces an open safety platform using OpenShell and Sentry to prevent AI agents from deviating from intended tasks and ensure security.
In a significant move to secure the future of autonomous computing, NVIDIA has unveiled an open safety platform specifically designed for AI agents. This new system introduces monitoring and security controls that exist entirely outside the AI model itself, ensuring that autonomous systems do not stray from their intended boundaries or tasks.
The NVIDIA Open Agent Safety Platform integrates two core technologies: OpenShell and Sentry. By combining these, NVIDIA aims to create an independent layer capable of policy enforcement, constant monitoring, and robust security for AI agents operating in real-world environments.
The Challenge of AI "Drift"
NVIDIA's research highlights a critical vulnerability in autonomous systems known as "drift." This occurs when an AI agent's actions begin to depart from its original instructions or operating constraints. According to the company, drift can be triggered by several factors, including:
- Software bugs or missing tools.
- Policy blocks or ambiguous instructions.
- The cumulative effect of agents working on complex problems for extended periods.
Crucially, NVIDIA noted that this behavior cannot simply be "trained away" without compromising the agent's capabilities. The company emphasized a vital lesson learned: "an agent in these circumstances cannot be expected to fully govern its own behaviour." This realization led to the development of security controls that remain outside the agent's reach, preventing the AI from tampering with its own restrictions.
How OpenShell and Sentry Work Together
To combat these risks, NVIDIA is utilizing a multi-layered approach to create a "trust layer" for the ecosystem:
OpenShell: An Open-Source Secure Runtime
OpenShell provides a sandboxed environment with kernel-level isolation. It functions by converting an operator's instructions into a verifiable policy. This allows users to strictly define which files, networks, tools, processes, and credentials an agent is permitted to access. OpenShell performs checks both before an agent begins operating and continuously while it is running.
NVIDIA Sentry: Hardware-Level Protection
For organizations requiring even deeper security, NVIDIA Sentry extends monitoring into NVIDIA BlueField hardware. By utilizing DOCA software, Sentry makes the BlueField security foundation programmable and links it directly with OpenShell policies. This creates a contextual record of all agent activities, including interactions, policy decisions, and tool access, making it easier to identify suspicious behavior or behavioral drift.
Five Principles for Safer AI Systems
NVIDIA has outlined a strategic framework for building safer agentic systems, centered on five key principles:
- Verifiable Policies: Ensuring all rules can be checked and validated.
- External Enforcement: Keeping security controls outside the agent's control.
- Controlled Access: Managing the path leading to the AI model.
- Scalable Authority: Allowing agent authority to grow only alongside the ability to inspect its behavior.
- Shared Responsibility: Applying a model of accountability across AI labs, enterprises, and infrastructure providers.
NVIDIA CEO Jensen Huang stated that the company is already working with more than 100 industry partners on this platform. "This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems," Huang remarked.
The platform is designed to run on NVIDIA Vera CPU and BlueField DPU-based systems, though it remains compatible with other hardware. This architecture allows organizations to manage entire fleets of agents and sub-agents within a strictly controlled boundary, ensuring continuous, out-of-band monitoring throughout their entire operation.