Senior Deep Learning Software Engineer, Inference

What you'll do

• Design, build, and optimize GPU-accelerated software for large-scale AI model serving and inference.
• Develop and maintain high-performance open-source inference frameworks and libraries.
• Implement and optimize model serving pipelines using CUTLASS, OAI Triton, NCCL, and CUDA kernels.
• Identify and drive performance improvements for LLM and Generative AI models across NVIDIA accelerators.
• Contribute features and code to NVIDIA’s inference libraries, including vLLM, SGLang, and FlashInfer.
• Implement the latest deep learning algorithms for public release in inference frameworks.
• Scale DL model performance across various architectures from datacenter GPUs to edge SoCs.

What we're looking for

• Master's or PhD or equivalent experience in Computer Engineering, Computer Science, EECS, or AI.
• 5+ years of relevant software development experience.
• Excellent C/C++ programming and software design skills.
• Python experience (preferred).
• Prior experience with training, deploying, or optimizing the inference of DL models in production (preferred).
• Prior background with performance modeling, profiling, debugging, code optimization, or architectural knowledge of CPU and GPU (preferred).
• GPU programming experience with CUDA, OAI TRITON, or CUTLASS (preferred).
• Experience with Multi GPU Communications (NCCL, NVSHMEM) (preferred).