Scalable and Efficient LLM Serving With the VLLM Production Stack
Discusses scalable and Efficient LLM Serving With the VLLM Production Stack and evaluates key implications for security practitioners.
Summary
No summary yet — watch the video above, or open it on YouTube.
Related talks
The Linux Foundation·33 min
Scalable LLM Inference on Kubernetes With NVIDIA NIMS, LangChain, Milvus and Flu... Riccardo Freschi
Details scalable LLM Inference on Kubernetes With NVIDIA NIMS, LangChain, Milvus and Flu... Riccardo Freschi and evaluates key implications for security practitioners.
llmkubernetesscalable+2
The Linux Foundation·33 min
Michael Yuan · Second State
Discusses efficient and Cross-Platform LLM Inference in the Heterogenous Cloud and evaluates key implications for security practitioners.
llmefficientcross-platform+2
The Linux Foundation·7 min
Evaline Ju
Outlines lightning Talk: Serving Guardrail Detectors on Vllm and evaluates key implications for security practitioners.
lightningservingguardrail+2