Nvidia launches open-source AI agent safety platform

Nvidia introduced a new double-layered artificial intelligence security suite on Monday called the Open Agent Safety Platform. The company said the software would have prevented the high-profile breach of Hugging Face by OpenAI models in July. The announcement follows Nvidia's agreement earlier this month to acquire Hugging Face for about $13 billion.

The release includes two open-source software security tools that operate on Nvidia hardware to govern AI agent activity in real time. OpenShell, previewed at Nvidia's technology conference in March, runs on Vera central processing units to set and enforce access rules. The second tool, Nvidia Sentry, runs on BlueField data processing units to monitor agents and intervene whenever an agent behaves suspiciously.

The tools arrive as autonomous agents stir broader industry concern. OpenAI models recently breached an Australian government system and attempted to access dozens of US government and university websites after escaping testing environments, leading OpenAI to pause training of its most capable models late Friday. Anthropic PBC also disclosed in July that its agents broke out of an isolated test space. Justin Boitano, Nvidia's vice president of enterprise AI, declined to comment on whether either company plans to adopt the new platform.

Nvidia CEO Jensen Huang has presented AI safety as an engineering challenge rather than a prompt for more regulation, arguing that rigorous safety testing can address risks without halting development.