As a senior AI infrastructure software engineer on NVIDIA’s DGX Cloud AI Efficiency Team, you will play a pivotal role in developing and maintaining the tools that optimize efficiency and resiliency for large-scale AI workloads. Your responsibilities include implementing robust software solutions to ensure high availability of AI systems, co-designing APIs with NVIDIA's resiliency stacks, and enhancing infrastructure underpinning AI platforms. You will also define reliability metrics to track system performance. Ideal candidates have over 8 years of experience in building scalable distributed systems for AI, proficiency in Python, C/C++, and scripting languages, and expertise in observability tools like ELK, Prometheus, and Loki. Additionally, familiarity with RDMA software stacks such as NCCL and ucx is beneficial. This role offers the chance to work on cutting-edge technologies that drive advancements in AI and data science within a collaborative environment focused on iterative improvement and risk-taking.
Develop infrastructure software and tools for large-scale AI pre-training, post-training, and inference.
Optimize tools and libraries to enhance efficiency and resiliency of AI workloads.
Co-design and implement APIs for integration with NVIDIA's resiliency stacks.
Enhance infrastructure underpinning NVIDIA’s AI platforms to support scalability and reliability.
Define and track reliability metrics to improve system and service stability.
Analyze failures from application level down to hardware level for root cause identification.
What we're looking for
Minimum 8+ years experience in developing software infrastructure for large-scale AI systems
Strong debugging and root cause analysis skills across application to hardware levels
Experience with observability platforms like ELK, Prometheus, and Loki
Proven track record in building and scaling distributed systems
Proficiency in Python, C/C++, and scripting languages
Expertise in quality software engineering practices including testing and CI/CD
Background in working with large-scale clusters and RDMA software stacks
Market check
Salary context
Above market
How this pay compares to similar roles
Similar $199k
This role $236k
$143k
most similar roles pay here
$303k
This role pays more than
77%
of similar roles. Most pay
$162,000–$235,750
— the shaded band above.
At the midpoint, this role pays about
$236k
versus about
$199k
for comparable roles.
Based on 239 similar postings.
Employer
About Nvidia
Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing
Nvidia currently has
824 open roles
on FindRole.
Listed pay typically runs
$184,000–$287,500
across 812 roles with salary data.