below marketmost similar roles pay hereabove market
This role pays more than
96%
of similar roles. Most pay
$175,662–$241,750
— the blue band above.
At the midpoint, this role pays about
$290k
versus about
$209k
for comparable roles.
Based on 240 similar postings.
Employer
About Nvidia
Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing
Nvidia currently has
1623 open roles
on FindRole.
Listed pay typically runs
$184,000–$287,500
across 1110 roles with salary data.
The Senior Deep Learning Software Infrastructure Engineer joins the Autonomous Vehicles project to build and scale training libraries and infrastructure for end-to-end autonomous driving models. You will develop robust training pipelines and frameworks to enable training on multi-thousand GPU clusters using massive video datasets. Key responsibilities include improving efficiency across data loaders, distributed training, scheduling, and performance monitoring while owning core components like orchestration libraries and fault-resilient systems. You will utilize Python and deep learning frameworks like PyTorch, leveraging knowledge of DDP, FSDP, NCCL, and tensor or pipeline parallelism. Technical requirements include experience with datacenter networking like RoCE and IB, parallel filesystems such as Lustre, and schedulers like Slurm or Kubernetes. You will collaborate with researchers to minimize stalls and improve training availability.
Craft and scale deep learning infrastructure libraries for training on multi-thousand GPU clusters.
Improve efficiency across the training stack, including data loaders, distributed training, and performance monitoring.
Build robust training pipelines to handle massive video datasets and enable rapid experimentation.
Own core infrastructure components such as orchestration libraries and fault-resilient training systems.
Minimize training stalls and improve system availability through close collaboration with research and platform teams.
Ensure infrastructure scales with growing GPU capacity and dataset sizes while maintaining developer stability.
Translate research requirements into production-grade Python libraries and automation tools.
What we're looking for
BS, MS, or PhD in Computer Science, Electrical/Computer Engineering, or a related field, or equivalent experience.
12+ years of professional experience building and scaling high-performance distributed systems, ideally in ML, HPC, or large-scale data infrastructure.
Extensive knowledge in deep learning frameworks (PyTorch is preferred), large scale training (DDP/FSDP, NCCL, tensor/pipeline parallelism), and performance profiling.
Strong systems background in datacenter networking (RoCE, IB), parallel filesystems (Lustre), storage systems, and schedulers (Slurm, Kubernetes, etc.).
Proficiency in Python with experience writing production-grade libraries, orchestration layers, and automation tools.
Ability to work closely with multi-functional teams to translate requirements into robust systems.
Shown experience scaling large GPU training clusters with >1,000 GPUs (preferred).
Expertise in fault resilience, high availability, elastic training, and large-scale observability (preferred).
Remote (Santa Clara, CA)+428 days ago
$184,000–$287,500
Python
C/C++
PyTorch
TensorFlow
JAX
Ray
Distributed Systems
RDMA
NCCL
IB Verbs
UCX
Libfabrics
Prometheus
ELK
Loki
CI
Version Control
High-Performance Computing
APIs
PyTorch
Python
C++
Distributed Training
NCCL
DDP
FSDP
Kubernetes
Slurm
RoCE
Lustre
Deep Learning
Performance Profiling
Reinforcement Learning
Fault Resilience
PyTorch
JAX
TensorFlow
Python
C++
CUDA
Kubernetes
SLURM
Reinforcement Learning
Deep Learning
MLOps
Distributed Training
LLMs
Sim-to-Real
HPC
Natural Language Processing
Graphics