Senior Software Engineer, AI Inference Performance

What you'll do

• Analyze end-to-end LLM and VLM inference processes to optimize latency, throughput, and processing efficiency.
• Build speed-of-light and roofline models to quantify performance headroom and identify optimization hypotheses.
• Profile workloads using tools like NVIDIA Nsight Systems and PyTorch Profiler to eliminate bottlenecks in host code and CUDA kernels.
• Tune serving hyperparameters including batching, KV-cache management, quantization, and speculative decoding.
• Develop and optimize performance-critical kernels for attention, matrix multiplication, and mixture-of-experts routing using CUDA or Triton.
• Establish repeatable benchmarks and performance regression gates to ensure consistent production quality.
• Contribute high-quality code improvements to open-source inference engines like TensorRT-LLM, vLLM, and SGLang.

What we're looking for

• More than 6 years of experience in full-stack LLM/VLM inference performance involving models, serving, distributed runtimes, kernels, and hardware.
• Strong programming skills in Python, Rust, and/or C++.
• Hands-on experience with CUDA or another GPU programming environment.
• Demonstrated expertise in speed-of-light analysis, roofline models, microbenchmarks, and tools including NVIDIA Nsight Systems and Nsight Compute.
• Deep understanding of GPU architecture, including Tensor Cores, memory hierarchy, caches, occupancy, synchronization, and numerical formats.
• Understanding of distributed systems and networking for accelerated computing, including collectives, topology, and scale-up/scale-out performance.
• BS or MS in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• Contributions to high-performance AI projects like TensorRT-LLM, vLLM, SGLang, PyTorch, CUDA, Triton, or NCCL (preferred).