This role pays more than
85%
of similar roles. Most pay
$208,600–$270,000
— the shaded band above.
At the midpoint, this role pays about
$290k
versus about
$239k
for comparable roles.
Based on 240 similar postings.
Employer
About Nvidia
Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing
Nvidia currently has
896 open roles
on FindRole.
Listed pay typically runs
$184,000–$287,500
across 876 roles with salary data.
As an Engineering Manager, LLM Inference & Deployment at Scale, you will lead and mentor a high-performing team responsible for building and operating an AI inference platform. You will manage the transition of Large Language Models and Vision-Language Models from research prototypes into optimized, reliable, and scalable production deployments. Your daily work involves driving technical strategy for low-latency, high-throughput, and cost-efficient inference while establishing engineering standards for benchmarking and profiling. You will utilize expertise in quantization, speculative decoding, continuous batching, prefix caching, and KV-cache optimization to improve performance across distributed systems and GPU hardware. The role requires proficiency with tools like TensorRT, TensorRT-LLM, vLLM, and SGLang. You will solve complex problems at the intersection of model optimization, inference systems, and production infrastructure to enable seamless deployment of rapidly evolving generative AI models.
What does a Engineering Manager earn in California?
Median $290250 from 64 postings across 19 companies.
LLM
PyTorch
Python
Triton-Inference Server
vLLM
SGLang
CUDA
Triton
torch.compile
torchDynamo
Distributed Systems
Kernel Design
Deep Learning
KV-Cache Management
Model Optimization
CUDA
Python
C++
Rust
TensorRT-LLM
vLLM
SGLang
Triton
PyTorch
NVIDIA Nsight Systems
CUTLASS
NCCL
Quantization
Distributed Systems
GPU Architecture
Model Parallelism
Speculative Decoding