Senior Software Engineer, AI and DL Kernel Libraries

Nvidia

Actively hiring
Remote (Us, Ca, Santa Clara, US) Posted 36 days ago $184,000–$287,500 / year
View original post Log in to save

At a glance

AI generated

TL;DR

Join our dynamic team as an AI Systems Engineer at NVIDIA, where you will play a pivotal role in developing cutting-edge technologies for the inference systems software stack. Your responsibilities include innovating and building new libraries, code generators, and GPU kernel technologies to optimize large language models and other high-impact AI workloads. You’ll collaborate closely with cross-functional teams across deep learning frameworks, libraries, and GPU architecture groups while contributing to open-source communities like FlashInfer, vLLM, and SGLang. Ideal candidates possess a master’s degree in Computer Science or Electrical Engineering (PhD preferred), along with extensive experience in ML/DL systems development, Python and C/C++ programming, and GPU kernel optimization using CUDA C/C++. Expertise in domain-specific compilers for LLM inference, machine learning compilers like Apache TVM, and open-source project contributions are highly valued.

Skills

Python C/C++ CUDA C/C++ cuTile Triton FlashInfer vLLM SGLang Apache TVM MLIR PyTorch JAX TensorFlow ONNX

What you'll do

  • Design and implement new abstractions for LLM serving engines.
  • Develop efficient attention kernel implementations to accelerate AI workloads.
  • Build just-in-time domain-specific compilers and runtimes for AI inference.
  • Optimize GPU kernels for NVIDIA's hardware architecture using CUDA C/C++.
  • Contribute to open source communities like FlashInfer, vLLM, and SGLang.

What we're looking for

  • Masters degree in Computer Science, Electrical Engineering, or related field; PhD preferred.
  • 6+ years of experience in ML/DL systems development.
  • Strong expertise in deep learning frameworks and inference engines like PyTorch, vLLM, SGLang.
  • Proficient in Python and C/C++ programming.
  • Extensive GPU kernel development and performance optimization skills using CUDA C/C++, Triton.
  • Background in domain-specific compiler solutions for LLM inference and training.
  • Expertise in machine learning compilers such as Apache TVM, MLIR.

Market check

Salary context

This $184,000–$287,500 range sits above 82% of similar postings on FindRole.

Peer median band

$152,000–$241,500

Median floor and ceiling across peers.

Typical midpoint (25–75%)

$159,750–$235,750

Middle half of comparable postings.

Based on 240 comparable postings.

* 240 is the maximum number of comparable postings sampled.

Employer

About Nvidia

Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing

Nvidia currently has 802 open roles on FindRole.

Listed pay typically runs $184,000–$287,500 across 798 roles with salary data.

Most-posted roles

View all roles at Nvidia

More like this

Similar roles

AI Software Engineer, Kernel Libraries - New College Grad 2026

Nvidia

Us, Ca, Santa Clara, US 9 days ago $124,000–$195,500
Python C++ CUDA cuTelemetry Triton PyTorch JAX TensorFlow ONNX vLLM SGLang MLIR FlashInfer Apache TVM NVIDIA GPU Architecture Domain Specific Compilers Open Source Contributions

Senior Software Engineer, AI Agent Runtime and Open Source Infrastructure

Nvidia

Us, Ca, Santa Clara, US 17 days ago $224,000–$356,500
TypeScript Node.js Rust Docker Kubernetes CI/CD Python Linux GPU PostgreSQL Terraform AWS OpenShell AI_platforms LLM_inference network_policy_management security_conscious_engineering containers Linux_isolation_technologies

Senior Software Engineer - ML Infrastructure

Plaid

San Francisco Hq, US 36 days ago $190,800–$262,800
MLFlow SageMaker Python Kubernetes Terraform AWS CI/CD PostgreSQL Docker Prometheus Grafana GitLab LLMs model registries

Senior Software Engineer - ML Infrastructure

Plaid

New York City Office, US 36 days ago $190,800–$262,800
MLFlow SageMaker Python Kubernetes Terraform CI/CD PostgreSQL Docker Prometheus Grafana AWS GitLab LLMs model registries

Senior Software Engineer - ML Infrastructure

Plaid

Seattle Office, US 36 days ago $190,800–$262,800
MLFlow SageMaker Python Kubernetes Docker CI/CD Prometheus Grafana PostgreSQL AWS Terraform Git GitHub Jenkins Ansible Kafka Redis MongoDB GraphQL