Principal Software Engineer, E2E Performance and Goodput

What you'll do

• Act as the technical focal point for end-to-end performance with key CSP and hyperscale customers.
• Lead performance characterization work streams to educate customers on platform expectations, profiling methods, and tuning options.
• Synthesize customer feedback to identify performance gaps and advocate for optimization priorities within internal engineering teams.
• Update and validate open-source performance and stress tools for the latest NVIDIA rack-scale systems and architectures.
• Analyze cross-CSP performance patterns to identify software or configuration issues causing performance gaps.
• Ensure customer performance and validation tooling reflects current GPU capabilities and memory hierarchy changes.
• Coordinate with customers to ensure profiling infrastructure and benchmark harnesses are ready before deployment milestones.
• Define test strategies and tooling requirements for both internal certification and customer acceptance.

What we're looking for

• 15+ years of experience in systems performance engineering, ideally in GPU/HPC/ML infrastructure.
• BS or MS in Computer Science, Computer Engineering, or a related field (or equivalent experience).
• Proficiency in GPU workload profiling using tools like Nsight Systems, Nsight Compute, and DCGM metrics.
• Understanding of distributed training performance dynamics including computation/communication overlap and memory bandwidth utilization.
• Knowledge of how the full software stack impacts performance, including driver overhead, firmware power management, and scheduling.
• Strong data analysis and visualization skills using Python, pandas, and dashboards.
• Proficiency in statistical methods for performance analysis such as regression detection and A/B comparison at scale.
• Ability to communicate complex technical findings to both deep technical audiences and executive leadership.