Senior Deep Learning Software Engineer, Inference

Nvidia

Confirmed live today High trust

Remote

Quick summary

Work type: Remote
Location: Remote
Salary: $184,000–$287,500 / yr
Employment: Full-time
Posted: 17 days ago
Freshness: Confirmed live today

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $220k

This role $236k

$162k most similar roles pay here $301k

This role pays more than 65% of similar roles. Most pay $188,987–$251,875 — the shaded band above. At the midpoint, this role pays about $236k versus about $220k for comparable roles.

Based on 240 similar postings.

Employer

About Nvidia

Nvidia is a leading designer of graphics processing units (GPUs) and system-on-chip units, powering gaming, professional visualization, data centers, and artificial intelligence workloads. Industry: Semiconductors & AI Computing

Nvidia currently has 892 open roles on FindRole.

Listed pay typically runs $184,000–$287,500 across 870 roles with salary data.

Most-posted roles

View all roles at Nvidia

At a glance

TL;DR · Senior Deep Learning Software Engineer, Inference

Role Posting Log in to save

Senior Deep Learning Software Engineer, Inference will join the team to design, build, and optimize GPU-accelerated software powering sophisticated AI applications. The role focuses on developing high-performance deep learning frameworks, specifically SGLang and vLLM, to facilitate the deployment of large language models. You will perform performance optimization, analysis, and tuning for LLM, Multimodal, and Generative AI models across various NVIDIA accelerators from datacenter GPUs to edge SoCs. Key responsibilities include contributing code to inference libraries like FlashInfer and collaborating with cross-functional teams on innovative solutions. Required skills include expert C/C++ programming, software design, and Python. You will utilize tools such as CUTLASS, OAI Triton, NCCL, and CUDA kernels to optimize model serving pipelines. The work addresses the technical challenge of efficient large-scale model serving and high-performance inference for state-of-the-art generative models.

Skills

CUDA C++ Python vLLM SGLang PyTorch Triton CUTLASS NCCL NVSHMEM FlashInfer Deep Learning Generative AI LLM GPU Programming Performance Optimization High-Performance Computing

What you'll do

Design, build, and optimize GPU-accelerated software for high-performance deep learning inference.
Develop and maintain core inference libraries including SGLang, vLLM, and FlashInfer.
Implement the latest deep learning algorithms for public release in open-source frameworks.
Identify and drive performance improvements for LLM and Generative AI models across NVIDIA accelerators.
Optimize model serving pipelines using tools like CUTLASS, OAI Triton, NCCL, and CUDA kernels.
Scale the performance of deep learning models across various hardware architectures from datacenter GPUs to edge SoCs.
Perform performance modeling, profiling, debugging, and code optimization for production inference systems.

What we're looking for

Master's or PhD degree in Computer Engineering, Computer Science, EECS, or AI (or equivalent experience).
6+ years of relevant software development experience.
Excellent C/C++ programming and software design skills.
Experience with Python is a plus.
Prior experience training, deploying, or optimizing the inference of DL models in production is a plus.
Background in performance modeling, profiling, debugging, code optimization, or CPU/GPU architectural knowledge is a plus.
Contributions to Deep Learning Software projects like PyTorch, vLLM, and SGLang are preferred.
Experience with Multi-GPU Communications (NCCL, NVSHMEM) and GPU programming (CUDA, OAI TRITON, or CUTLASS) is preferred.

Save