Manager, Software Engineering, Production AI Inference

Nvidia

Quick summary

Work type
On-site
Location
Santa Clara, CA
Salary
$224,000–$356,500 / yr
Posted
1 day ago
Nearby
99+ roles within 25 mi

Market check

Salary context

Above market

How this pay compares to similar roles

Similar $195k
This role $290k
$126k most similar roles pay here $381k

This role pays more than 95% of similar roles. Most pay $159,093–$230,662 — the shaded band above. At the midpoint, this role pays about $290k versus about $195k for comparable roles.

Based on 240 similar postings.

Employer

About Nvidia

Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing

Nvidia currently has 928 open roles on FindRole.

Listed pay typically runs $184,000–$287,500 across 915 roles with salary data.

Most-posted roles

View all roles at Nvidia

At a glance

TL;DR · Manager, Software Engineering, Production AI Inference

Role Posting Log in to save

As a senior software manager at NVIDIA, you will lead the team responsible for transforming cutting-edge AI models into reliable, production-ready inference microservices (NIM) across diverse environments. Your role involves hands-on management of engineers working on model onboarding, performance optimization, release quality assurance, and operational health, ensuring seamless integration with other internal teams like product and security to enhance NIM’s reliability. You will establish a predictable operating model through strategic planning and continuous improvement initiatives, while also building and mentoring a world-class AI inference engineering team. The ideal candidate has over 10 years of experience in production software development, including extensive management expertise, deep knowledge of AI/ML fundamentals, and hands-on experience with GPU technologies such as CUDA and cuDNN. Additionally, familiarity with large-scale distributed systems, security hardening, and enterprise deployment requirements is crucial for this role.

Skills

CUDA cuDNN Kubernetes TensorRT PyTorch Triton FedRAMP PostgreSQL CI/CD Prometheus Grafana GitLab AWS NVIDIA_TensorRT vLLM SGLang Dynamo Python

What you'll do

  • Lead team responsible for shipping production-ready LLM NIMs through planning and execution.
  • Build predictable operating model with roadmap planning, weekly execution rhythm, and clear ownership boundaries.
  • Own project execution by managing risks and adapting plans to keep engineering timelines agile.
  • Drive continuous improvement in production workflows through RCA and partner feedback.
  • Mentor engineers and emerging leaders while fostering an innovative culture.

What we're looking for

  • 10+ years of building production software with at least 3 years in managing engineering teams.
  • Deep expertise in AI/ML fundamentals, model architectures, inference engines, performance optimization, accelerated computing, and large-scale distributed systems.
  • Proven track record of delivering high-quality, reliable production software releases.
  • Experience driving process improvements to enhance operational efficiency and team productivity.
  • Strong communication skills with the ability to influence executive leadership across various departments.
  • Hands-on experience with core GPU technologies like CUDA, cuDNN, and NVLink for performance optimization.

Related searches

More Manager jobs Manager jobs in Santa Clara All jobs at Nvidia

More like this

Similar roles

Engineering Manager, AI Developer Tools

Apple Inc

Seattle, WA 75 days ago $195,700–$338,400
Python Java CI/CD DORA SPACE Kubernetes Terraform PostgreSQL MongoDB LangChain Pydantic AI PyTorch TensorFlow Hugging Face Kafka Messaging Systems Microservices Architectures Prometheus Grafana
Save

AI Native Software Engineering Manager

Accenture

Arlington, VA +4 17 days ago $94,400–$305,000
Python Kubernetes Docker CI/CD Terraform Helm PostgreSQL OpenAI Anthropic Google Vertex AI RAG Prometheus Grafana Java
Hybrid
Save

Lead Software Engineer, AI

JPMorgan Chase

Columbus, OH 14 days ago
Python TypeScript LangChain LlamaIndex AutoGen CrewAI Cloud Foundry GKP Jules CI/CD TrueCD Sophia auth LLM APIs Kubernetes Docker CI/CD Prometheus Grafana
Save

Engineering Manager, AI Developer Technology

Nvidia

Santa Clara, CA +4 108 days ago $224,000–$356,500
CUDA C/C++ Python GPU CPU MPI OpenMP pthread Linear Algebra Deep Learning Machine Learning Parallel Programming Algorithm Optimization NVIDIA Platforms
Hybrid
Save