below marketmost similar roles pay hereabove market
This role pays more than
78%
of similar roles. Most pay
$142,400–$214,000
— the blue band above.
At the midpoint, this role pays about
$219k
versus about
$178k
for comparable roles.
Based on 240 similar postings.
Employer
About Nvidia
Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing
Nvidia currently has
1693 open roles
on FindRole.
Listed pay typically runs
$184,000–$287,500
across 669 roles with salary data.
Software Engineer, Performance Instrumentation will join the team to define and maintain performance data requirements from profiling libraries, annotation APIs, frameworks, runtimes, communication libraries, schedulers, configurations, and infrastructure telemetry. The role involves identifying visibility gaps, building low-overhead instrumentation, and ensuring data quality through validation, sanitization, and alerting mechanisms. Key responsibilities include measuring instrumentation overhead on real workloads and collaborating with partner teams to manage commitments. The position requires proficiency in Python, C++, and CUDA, along with working knowledge of machine learning frameworks like PyTorch or JAX and tracing APIs such as NVTX. The work focuses on production machine learning and systems software, specifically addressing the technical challenges of telemetry conventions, storage, checkpointing, and network or collective-communication telemetry including NCCL and fabric counters.
What does a Software Engineer earn in California?
Median $214500 from 1134 postings across 74 companies.
Define and maintain performance data requirements from profiling libraries, frameworks, runtimes, and infrastructure telemetry.
Identify and prove visibility gaps with concrete evidence to determine which diagnoses are currently impossible.
Build low-overhead instrumentation and annotation APIs that efficiently convey necessary information to consumers.
Implement validation, sanitization, and alerting mechanisms to ensure data quality at the ingestion point.
Measure and manage the overhead and observer effects of instrumentation on real-world workloads.
Manage commitments and ensure continuous progress when collaborating with multiple partner teams simultaneously.
Implement code modifications in other groups' projects by identifying requirements and reaching consensus on interfaces.
What we're looking for
Bachelor’s degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
8+ years of relevant software engineering experience, including hands-on experience with instrumentation or tracing in production machine learning or systems software.
Proficiency in CUDA and working knowledge of machine learning frameworks like PyTorch or JAX.
Proficiency in Python and C++.
Experience with annotation or tracing APIs such as NVTX.
Demonstrated experience in maintaining data quality in a pipeline, including validation, checks, alerting, and provenance.
Proven ability to measure instrumentation overhead and make trade-offs among overhead, fidelity, sampling, and aggregation.
Experience establishing telemetry conventions or schemas adopted by other teams (preferred).