This role pays more than
99%
of similar roles. Most pay
$151,000–$235,750
— the shaded band above.
At the midpoint, this role pays about
$290k
versus about
$193k
for comparable roles.
Based on 240 similar postings.
Employer
About Nvidia
Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing
Nvidia currently has
896 open roles
on FindRole.
Listed pay typically runs
$184,000–$287,500
across 876 roles with salary data.
As a Senior Software Engineer - GPU Local AI Platforms, you will join the Local AI team to build the software stack enabling large language models and generative AI applications to run efficiently on edge hardware. You will track innovations in open-source LLM inference frameworks, analyze how new architectures like MoE routing or speculative decoding map onto GPU architecture, and characterize multi-node inference behavior using NCCL/RCCL. Your daily work involves producing performance analysis reports, owning model validation workflows, and maintaining developer-facing inference recipes. The role requires expertise in Python, C++, CUDA, Triton, and container engineering with Docker. You will solve technical challenges regarding memory bandwidth, throughput, and latency while serving as a technical point of contact for hardware-specific issues to ensure community innovations are reliable at scale for developers and partners.
What does a Software Engineer earn in California?
Median $214000 from 775 postings across 63 companies.
Evaluate innovations in open-source LLM inference frameworks to identify performance-critical features for edge hardware.
Analyze how new model architectures and inference algorithms map onto NVIDIA GPU architecture for optimization.
Characterize multi-node inference behavior including collective communication primitives and parallelism efficiency on edge clusters.
Produce performance analysis reports mapping theoretical hardware limits to observed inference throughput, latency, and utilization.
Manage the model validation workflow for new releases, including compatibility assessment and recipe development.
Develop and maintain accurate developer-facing inference recipes while automating staleness detection from CI results.
Serve as the technical point of contact for partners regarding hardware-specific inference issues.
What we're looking for
BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience.
12+ years of software engineering experience with depth in GPU computing, ML systems, or high-performance inference.
Strong Python or C++ programming skills and expertise in software design and engineering.
Hands-on experience with GPU kernel development or optimization using CUDA, C++, Triton, or equivalent.
Working knowledge of LLM inference internals including attention mechanisms, KV-cache management, continuous batching, quantization, and tensor parallelism.
Expertise in container engineering including multi-architecture Docker/OCI builds and NVIDIA Container Toolkit.
Strong analytical skills to form performance hypotheses, design experiments, and communicate findings clearly.
CUDA
Python
C++
Rust
TensorRT-LLM
vLLM
SGLang
Triton
PyTorch
NVIDIA Nsight Systems
CUTLASS
NCCL
Quantization
Distributed Systems
GPU Architecture
Model Parallelism
Speculative Decoding