'Back to the Future' with self-hosted bare metal

### Overview In this episode of IT Ops Query, host Beth Pariso speaks with Rob Hirschfeld, CEO and co-founder of RackN, regarding the return to self-managed bare metal infrastructure and on-premises data centers. As enterprises grapple with unpredictable public cloud expenditures, VMware exit strategies following the Broadcom acquisition, and the immense demands of generative AI workloads, self-hosted hardware is re-emerging as a strategic priority. Hirschfeld outlines how contemporary bare metal operations diverge significantly from legacy 2005-era data centers, emphasizing modern automation, infrastructure-as-code, and robust operational processes.

### Key Topics - **Cloud Repatriation and Cost Realities:** Enterprises with cloud-first strategies are finding cloud expenses harder to forecast and control. For stable workloads, purchasing, depreciating, and owning physical assets often provides superior economic value. - **The Platform Engineering vs. Bare Metal Divide:** Modern platform engineering teams have largely developed an application-centric focus around Kubernetes and hosted cloud environments. Hirschfeld points out that these teams frequently lack the lower-level storage, networking, and hardware expertise needed to run physical infrastructure or execute a VMware exit. - **The Challenge of VMware Exits:** Migrating away from VMware to alternatives like OpenShift virtualization requires extensive bare metal lifting. VMware offers decades of deep, standardized integration in advanced storage and networking that cloud-native orchestration frameworks do not fully replicate out of the box. - **Modernizing Hardware Operations:** Self-managed infrastructure today does not mean reverting to manual scripts or USB drives. Practices such as rack-scale automation, API-driven out-of-band management (e.g., Dell iDRAC), GitOps, and repaving infrastructure are being applied directly to physical bare metal. - **Generative AI Infrastructure Pressures:** Generative AI does not introduce a fundamentally different workload type, but it imposes extreme operational demands. High-cost systems ($250,000 to $500,000 per server) require complex configurations, smart NICs, RoCE/InfiniBand fabrics, and rigorous firmware management down to SSDs. Lengthy resets or misconfigurations can result in massive downtime and depreciation costs. - **Data Sovereignty and Workload Co-mingling:** The cost of paying per token and the security risks associated with sending sensitive enterprise data to external third parties are pushing companies to bring AI inference on premises. In the long term, AI workloads and agentic systems are expected to co-mingle with standard business applications.

### Technical Details - **Hardware Configuration and Out-of-Band Management:** Modern bare metal depends on API-driven out-of-band controllers like Dell's iDRAC. Hirschfeld notes real-world edge cases, such as vendors unexpectedly making boot sequence settings read-only in firmware updates, breaking automated provisioning. - **AI Hardware Fabrics and Interconnects:** AI clusters demand tight integration of advanced NICs, RoCE (RDMA over Converged Ethernet), and InfiniBand to facilitate shared memory across nodes, rendering off-the-shelf OEM profiles insufficient without continuous patching and automation. - **Infrastructure as Code and Rack Repaving:** Utilizing declarative infrastructure-as-code patterns allows operators to tear down and rebuild entire racks programmatically, treating bare metal with cloud-native immutability.

### Takeaways - **Focus on Day-Two Operations First:** Rather than hand-crafting pilots that fail to scale, organizations must establish clear operational processes for updating, patching, and automating bare metal early on. - **Address the Skills Gap:** The industry faces a shortage of engineers experienced in physical infrastructure. Solving this requires standardizing operations and leveraging modular automation rather than relying on bespoke configurations or rigid converged vendor lock-in. - **Maintain Infrastructure Autonomy:** Controlling physical infrastructure protects sensitive data sovereignty, mitigates uncontrollable cloud pricing, and preserves core organizational autonomy.