NVIDIA Triton Inference Server and its use in Netflix's Model Scoring Service

This spring at Netflix HQ in Los Gatos, we hosted an ML and AI mixer that brought together talks, food, drinks, and engaging discussions on the latest in machine learning, infrastructure, LLMs, and foundation models.
This talk was by Amr Elmeleegy, NVIDIA, Fan Yang and Liping Peng, Netflix.