Engineering Manager, Deep Learning Inference

Nvidia

Confirmed live today Trusted

Hybrid

Quick summary

Work type: Hybrid
Location: Santa Clara, CAGADCILCAMA
Employment: Full-time
Posted: 91 days ago
Freshness: Confirmed live today
Nearby: 99+ roles within 25 mi

Market check

Salary context

How this pay compares to similar roles

Similar $243k

$172k $300k

below market most similar roles pay here above market

This listing doesn't post a salary. Most similar roles pay $208,631–$277,050.

Based on 240 similar postings.

Employer

About Nvidia

Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing

Nvidia currently has 1623 open roles on FindRole.

Listed pay typically runs $184,000–$287,500 across 1110 roles with salary data.

Most-posted roles

View all roles at Nvidia

At a glance

TL;DR · Engineering Manager, Deep Learning Inference

Role Posting Log in to save

JOB TITLE: Engineering Manager, Deep Learning Inference The Engineering Manager, Deep Learning Inference will lead a high-performing engineering team focused on advancing AI model deployment. You will guide the strategy, roadmap, and execution of open-source inference frameworks including SGLang, vLLM, and FlashInfer. Responsibilities include mentoring engineers, overseeing performance tuning, and optimizing large-scale models for LLM, multimodal, and generative AI applications. You will partner with internal compiler and research teams to deliver optimized inference pipelines across various accelerators. The role requires expertise in C/C++, Python, and GPU programming using CUDA, Triton, and CUTLASS. You must also manage multi-GPU communications involving NCCL and NVSHMEM. This position solves the technical challenge of making AI deployment scalable and efficient for real-time inference across datacenter clusters and edge devices.

What does a Engineering Manager earn in California?

Median $279400 from 84 postings across 24 companies.

See salary data

Skills

CUDA Triton CUTLASS C++ Python PyTorch vLLM SGLang TensorRT-LLM NCCL NVSHMEM NIXL FlashInfer Agile Deep Learning GPU Programming Distributed Inference

What you'll do

Lead and mentor a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
Guide the strategy, roadmap, and execution of NVIDIA's open-source inference frameworks.
Partner with internal compiler, libraries, and research teams to deliver optimized inference pipelines.
Oversee performance tuning, profiling, and optimization for LLM and multimodal generative AI applications.
Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications.
Represent the team in roadmap and planning discussions to align with broader AI and software strategies.

What we're looking for

MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.
Software development experience.
3+ years in technical leadership or engineering management.
Strong background in C/C++ software design and development.
Proficiency in Python (preferred).
Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
Proven record of deploying or optimizing deep learning models in production environments.
Experience leading teams using Agile or collaborative software development practices.

Save