Engineering Manager, LLM Performance

What you'll do

• Lead and grow a team responsible for pushing LLM inference performance across multiple frameworks.
• Drive the design, implementation, and optimization of key features for LLM inference.
• Improve inference performance on current and upcoming NVIDIA datacenter architectures and GPUs.
• Optimize the performance of important foundation models for inference.
• Work with benchmark teams to tune performance for key workloads.
• Integrate cutting-edge technologies to provide an intuitive developer experience for LLM deployment.
• Lead software development execution, including project planning, milestone delivery, and cross-functional coordination.

What we're looking for

• MS, PhD, or equivalent experience in Computer Science, Computer Engineering, AI, or a related technical field.
• Overall software engineering experience.
• 3+ years of technical leadership experience.
• Proven ability to lead and scale high-performing engineering teams across distributed and cross-functional groups.
• Strong background in C++ or Python with expertise in software design and delivering production-quality software libraries.
• Demonstrated expertise in large language models (LLM), vision language models (VLM), and/or inference in general.
• Deep understanding of GPU architecture, CUDA programming, and system-level performance tuning (preferred).
• Background in LLM inference or working with frameworks such as TensorRT-LLM, vLLM, or SGLang (preferred).