Beyond Round-Robin: GPU-Aware Load Balancing for LLM Inference in Kubernetes
Outlines beyond Round-Robin: GPU-Aware Load Balancing for LLM Inference in Kubernetes and evaluates key implications for security practitioners.
Summary
No summary yet — watch the video above, or open it on YouTube.
Related talks
The Linux Foundation·33 min
Scalable LLM Inference on Kubernetes With NVIDIA NIMS, LangChain, Milvus and Flu... Riccardo Freschi
Details scalable LLM Inference on Kubernetes With NVIDIA NIMS, LangChain, Milvus and Flu... Riccardo Freschi and evaluates key implications for security practitioners.
llmkubernetesscalable+2
The Linux Foundation·30 min
Meenakshi Kaushik · Jayanth Srinivasa
Provides securing LLMs in Kubernetes: Best Practices and evaluates key implications for security practitioners.
kubernetesbest-practicesllm+1
The Linux Foundation·27 min
Komei Nakamoto · Keith Moyer
Details GKE Hypercluster: Kubernetes TEEs for AI at Scale and evaluates key implications for security practitioners.
kuberneteshyperclusterkubernete+2