Enabling Rack-scale Confidential Computing Using Heterogeneous Trusted Execution Environment

Zhu Jianping presents HETD, a heterogeneous trusted execution environment designed to enable rack-scale confidential computing in data centers without chip-level modifications. The motivation addresses the need to protect pooled computing resources (CPUs, GPUs, TPUs, FPGAs) commonly distributed across modern data centers, where existing TEE solutions either require hardware changes or cannot efficiently support such architectures.

The HETD system architecture consists of a sealed chassis containing a security controller (SSC based on MCPU and FPGA), PCI Express fabric for resource management, multiple proxy microservice nodes with accelerators, and untrusted external server nodes. The design separates security functions (in SSC) from computation (in HETD enclaves running standard software like TensorFlow and GPU drivers), minimizing the trusted computing base.

Key technical components include elastic resource allocation via PCIe fabric's software-defined and flexible topology properties, secure enclave lifecycle management through secure reboot procedures that clear memory and reset accelerators, and data protection using AES-GCM encryption with separate message queues for ciphertext and plaintext areas. An FPGA-based cryptographic controller enforces isolation, with no direct shared space between components. Trust establishment uses a two-layer public key design (endorsement key and attestation key) with remote attestation and symmetric key negotiation.

The threat model considers a strong adversary controlling the software stack and capable of side-channel attacks via network traffic analysis. Physical snooping on the host memory bus is in scope but not within the sealed HETD box. Covert channels, attacks on cryptographic algorithms, and denial-of-service attacks are out of scope.

The prototype integrates two Xilinx PX 9797 boards, one SC node, four processor nodes, and four NVIDIA GPUs. Performance evaluation shows throughput overhead under 5% for most workloads, with batch-size-dependent latency overhead reaching 42.96% for inference and 18.54% for training at batch size 8. Encryption overhead is small and stable. Multiple-GPU scalability matches the baseline, demonstrating that elastic resource allocation can improve single-enclave performance without sacrificing horizontal scalability.

The system provides four security guarantees in a Platform-as-a-Service model: isolation from other enclaves on the same proxy node, isolation from past and future users, protection from untrusted hosts, and protection from cloud providers. The design resists side-bleed attacks on sealed FPGAs and cold boot attacks when unauthorized chassis opening takes longer than power-removal-based memory extraction.