The $300 Enterprise Lab: Democratizing Infrastructure Skills Wit... Derek Bowdle & Cameron Khorsandi

### Overview In this presentation, Derek Bowdle and Cameron Khorsandi introduce the concept of the "$300 enterprise lab," a physical, localized alternative to practicing cloud deployments and cluster administration in public cloud environments. The speakers highlight the problem of "bill shock"—the intimidating risk of receiving exorbitant cloud bills from managed instances, test queries, or forgotten compute instances—which discourages junior engineers and novices from freely experimenting. Using an aviation flight simulator analogy, they argue that engineers need an environment where failure is a feature rather than an expensive or career-threatening mistake. By pairing a small cluster of Raspberry Pis with a locally hosted AI agent acting as a senior architect/mentor, learners can master infrastructure concepts safely, privately, and affordably.

### Key Topics - **Cloud Bill Shock and Gatekeeping:** Public cloud free tiers act as marketing funnels that restrict complex, multi-node setups behind paywalls. Running clusters 24/7 in the cloud with control planes, ingress/egress, and large instances costs thousands of dollars per year. - **The Mentorship Gap:** Site Reliability Engineers (SREs) are costly and have limited time to guide junior engineers step-by-step through Slack. - **Architecture Parity over Raw Compute:** A homelab does not need to match the raw compute power of hyperscaler data centers; its purpose is achieving architectural parity to understand clustering topology, resource allocation, and failure modes. - **Open-Source Infrastructure Stack:** Building an enterprise-like cluster on local hardware using lightweight, open-source software packages. - **Mentorship Workflow with Local AI:** Using offline, local AI models (via tools like LM Studio) to guide configuration, review errors, and draft YAML manifests without API token fees or data/key leakage risks.

### Technical Details - **Hardware Architecture:** The physical lab consists of three Raspberry Pis (specifically Raspberry Pi 5 boards with memory ranging from 4 GB to 16 GB), SSD storage via Pi HATs (or micro-SD cards), a network switch, and a coordinating machine (such as a MacBook Air M4). - **Software Ecosystem:** - **Kubernetes / Clustering:** K3s is used as the lightweight Kubernetes distribution. - **Networking & Storage:** MetalLB functions as the local load balancer, while Longhorn or MinIO provide distributed storage. - **Code & Version Control:** Forgejo provides a self-hosted Git repository. - **Monitoring / Visualization:** kube-ops-view is used to visually track node status and cluster operations. - **Local AI:** LM Studio and OpenCode run locally on the host machine. In the demonstration, the Qwen 2.5 / Qwen 3 Coder 30B model was executed on an Apple M4 Mac Air. - **Self-Healing and Quorum Live Demonstration:** The presenters demonstrated high availability by running a website across active nodes with three control planes. They pulled the power plug on Pi 2. MetalLB and K3s recognized the node as "NotReady," instantly shifted the website workload to Pi 3, and maintained site availability. When plugged back in, the node rejoined the cluster. The speakers emphasized the rule of odd numbers for maintaining quorum in control planes. - **Mentorship Workflow Steps:** (1) Role setting via system prompts in LM Studio, (2) Designing cluster topology and recipes, (3) Generating YAML files and terminal commands while keeping the human in the loop as the executor/pilot, and (4) Iterating through feedback loops by feeding terminal errors back to the AI for diagnosis.

### Takeaways - Building muscle memory and operational confidence is best accomplished in a safe sandbox where "pulling the plug" has no financial or operational penalties. - Running local AI models eliminates API costs, prevents secret leakage (such as inadvertently exposed API keys), and enables offline work without an internet connection. - Local hardware setups utilizing lightweight open-source platforms give engineers a risk-free platform to learn real-world Kubernetes, self-healing, and infrastructure topology before managing production environments.