This role pays more than
75%
of similar roles. Most pay
$162,000–$236,703
— the shaded band above.
At the midpoint, this role pays about
$236k
versus about
$199k
for comparable roles.
Based on 240 similar postings.
Employer
About Nvidia
Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing
Nvidia currently has
1150 open roles
on FindRole.
Listed pay typically runs
$184,000–$287,500
across 912 roles with salary data.
As a Senior Solutions Architect, AI Inference, you will join the inference service provider team to address technical challenges in scaling token generation on finite resources. You will build and operate inference recipes using tools like NVIDIA Dynamo while distributing tasks among GPU workers to improve efficiency. Your daily responsibilities include accelerating inference pipelines with TensorRT-LLM, vLLM, and SGLang, as well as evangelizing devops best practices for managing Kubernetes clusters and compute fabrics. You will provide technical leadership to partners, developing reference architectures for large gpu clusters. The role requires expertise in disaggregated serving, multi-tier KV cache management, speculative decoding, quantization, and custom inference kernels. You will also work on full-stack agent design involving sandboxing, memory, and retrieval systems while providing mentorship to customers and internal teams to resolve complex infrastructure issues.
What does a Solutions Architect earn in California?
Median $235750 from 35 postings across 9 companies.
Develop and write reference architectures for large-scale GPU clusters to optimize token generation and performance.
Build and operate inference recipes using tools like NVIDIA Dynamo to improve task distribution across GPU workers.
Accelerate inference pipelines using frameworks such as TensorRT-LLM, vLLM, and SGLang.
Evangelize DevOps best practices for managing Kubernetes clusters, compute fabrics, and system observability.
Provide technical leadership and mentorship to customers during the deployment of disaggregated inference systems.
Advise C-level executives and engineers on scaling their business infrastructure and AI systems.
Translate product insights from partner engagements into improvements for the broader industry's technology stack.
What we're looking for
6+ years of experience in Solutions Architecture, Solution Engineering, Forward-Deployed Engineering, or similar roles driving customer engagements for distributed systems.
2+ years of experience with AI workloads on Kubernetes.
Experience with NVIDIA Dynamo, Triton Inference Server, or TensorRT-LLM for model optimization and serving.
Deep knowledge of modern inference best practices including disaggregated serving, multi-tier KV cache management, speculative decoding, quantization, and custom kernels.
Hands-on full-stack agent design experience with sandboxing, memory, retrieval systems, evaluation, skill design, and governance.
BS in CS/Engineering or equivalent experience.
Prior experience deploying NVIDIA inference technologies such as Dynamo, NIXL, and Grove (preferred).
Experience with model customization techniques like SFT, DPO, GRPO, RLVR, and data curation; contributions to relevant open-source projects (preferred).
Remote (Santa Clara, CA)+165 days ago
$184,000–$287,500
GPU
InfiniBand
RoCE
RDMA
NCCL
NVLink
NVSwitch
Kubernetes
Slurm
Python
Linux
High-Performance Computing
Distributed Systems
Shell Scripting
DCGM
Nsight Systems