Building Trusted AI Factories with NVIDIA Accelerated Confidential Computing

### Overview At the Confidential Computing Summit, Eric Bunost, Product Manager for Confidential Computing at NVIDIA, presented on building trusted AI factories using accelerated confidential computing. As AI evolves from exploratory assistance to autonomous, reasoning agents performing complex work, data centers are shifting toward distributed "AI factories" evaluated primarily on cost per token. Deploying autonomous agents demands security and trust from day one, prompting NVIDIA to introduce frameworks like OpenShell and accelerate hardware-enforced confidential computing across its GPU and CPU architectures.

### Key Topics - **Agent Security and NVIDIA OpenShell**: NVIDIA internally emphasizes agentic workflows but discovered that standard security and compliance tools do not fit autonomous agents. To manage agent behavior, NVIDIA introduced OpenShell—an open-source community project that enforces sandboxing and operational policies, restricting which system calls, file system paths, and network services an agent can access. NVIDIA also released a Secure Agent Workspace reference design. - **Primary Confidential Computing Use Cases**: Customer interest centers on two main areas: protecting proprietary AI model IP and weights (allowing model providers to deploy models onto third-party infrastructure where data resides), and safeguarding customer sensitive data from untrusted infrastructure operators. - **Architectural Generations and Hardware Trust**: NVIDIA's confidential computing spans Hopper, Blackwell, and Vera Rubin. The architecture extends the hardware root of trust from CPU to GPU through isolation and encryption (including CPU-to-GPU interconnects and NVLink between GPUs), complemented by zero-trust enforcement via composite or remote attestation. - **Certification and Deployment Models**: NVIDIA expanded its NVIDIA Certified Systems program to validate confidential computing readiness across OEM/ODM hardware (e.g., Dell, HPE). For workloads, deployments span Confidential Virtual Machines (CVMs) and upstream-developed Confidential Containers (using Kata, Trusty, and Kubernetes). Red Hat OpenShift is highlighted as the first commercial downstream partner supporting this container architecture.

### Technical Details - **Performance and Inference Optimizations**: Confidential computing is heavily tested for inference workloads (e.g., Qwen 3.5 on Intel TDX systems with SGLang and TensorRT-LLM). Testing demonstrates that confidential computing overhead in metrics like throughput, time-to-first-token (TTFT), and time-per-output-token remains minimal (typically single-digit percentages) when inference framework optimizations are enabled. - **Eliminating Bounce Buffers**: Current architectures rely on bounce buffer optimizations, which have been built into TensorRT-LLM and submitted via PRs to upstream SGLang and FlashInfer (with vLLM under investigation). In contrast, the upcoming Vera Rubin NVL architecture eliminates bounce buffers entirely by leveraging TDIS (TEE Device Interface Security), delivering near-native performance out of the box. - **Vera Rubin Innovations**: The Vera Rubin NVL72 expands the trust boundary from up to 8 GPUs (in Blackwell) to an entire rack-scale boundary of 72 GPUs, designed for large frontier models. Additionally, Vera Rubin integrates the new Vera CPU, which is natively confidential computing capable.

### Takeaways Securing AI factories requires combining policy-based behavioral controls at the software layer (OpenShell) with hardware-rooted isolation and encryption across CPUs and GPUs. Through upstream container integration and hardware advancements like TDIS in Vera Rubin NVL72, confidential computing provides near-native performance while expanding trust boundaries to rack-scale deployments.