Engineering Manager, Deep Learning Inference

Nvidia

Confirmed live yesterday High trust

Hybrid

Quick summary

Work type: Hybrid
Location: Santa Clara, CADCTXNYWAMA
Salary: $184,000–$287,500 / yr
Employment: Full-time
Posted: 83 days ago
Freshness: Confirmed live yesterday
Nearby: 99+ roles within 25 mi

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $242k

This role $236k

$172k $300k

below market most similar roles pay here above market

This role pays less than 54% of similar roles. Most pay $206,212–$277,050 — the blue band above. At the midpoint, this role pays about $236k versus about $242k for comparable roles.

Based on 240 similar postings.

Employer

About Nvidia

Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing

Nvidia currently has 1626 open roles on FindRole.

Listed pay typically runs $184,000–$287,500 across 1115 roles with salary data.

Most-posted roles

View all roles at Nvidia

At a glance

TL;DR · Engineering Manager, Deep Learning Inference

Role Posting Log in to save

The Engineering Manager, Deep Learning Inference will lead a high-performing engineering team focused on advancing deep learning inference software and GPU-accelerated software. The role involves driving the strategy, roadmap, and execution of inference frameworks for Client AI, specifically focusing on large language models, multimodal, and generative AI applications. Key responsibilities include overseeing performance tuning, profiling, and optimization of large-scale models while mentoring engineers on best practices. The position requires expertise in C/C++, Python, and GPU programming using CUDA, Triton, and CUTLASS. Candidates must understand multi-GPU communications like NCCL and NVSHMEM to optimize inference pipelines across datacenter clusters and edge devices. The work involves developing and optimizing open-source frameworks such as vLLM, SGLang, and FlashInfer to make AI deployment scalable and efficient.

What does a Engineering Manager earn in California?

Median $277500 from 85 postings across 25 companies.

See salary data

Skills

CUDA Triton CUTLASS C++ Python vLLM SGLang FlashInfer TensorRT-LLM PyTorch NCCL NVSHMEM NIXL Multi-GPU Communications Distributed Inference Agile GPU Programming

What you'll do

Lead and mentor a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
Drive the strategy, roadmap, and execution of NVIDIA’s inference frameworks for Client AI.
Partner with internal compiler, libraries, and research teams to deliver optimized end-to-end inference pipelines.
Oversee performance tuning, profiling, and optimization of large-scale LLM and multimodal generative AI models.
Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications.
Represent the team in roadmap and planning discussions to align with broader AI and software strategies.

What we're looking for

MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.
Software development experience.
3+ years in technical leadership or engineering management.
Strong background in C/C++ software design and development.
Proficiency in Python (preferred).
Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
Proven record of deploying or optimizing deep learning models in production environments.
Experience leading teams using Agile or collaborative software development practices.

Save