Principal Software Engineer - AI Inference

What you'll do

• Drive upstream-first engineering in vLLM/SGLang by authoring and landing PRs.
• Build features to improve inference runtime efficiency, latency, and tail behavior.
• Optimize core hot paths across the stack for better GPU performance.
• Enhance multi-GPU and multi-node inference through improved communication patterns.
• Strengthen system correctness, robustness, and operability with observability hooks.
• Mentor senior engineers and establish guidelines for upstream contribution workflows.

What we're looking for

• 15+ years of experience in production software systems engineering with a track record of solving complex technical problems.
• Expertise in LLM inference/serving systems like vLLM and SGLang, understanding the trade-offs affecting real-world performance.
• Proficiency in Rust, C++, Python, CUDA for reading, modifying, and optimizing performance-critical code across layers.
• Experience with GPU performance analysis tools and methodologies including profiling, microbenchmarking, and memory/communication analysis.
• Strong background in distributed systems and concurrency, including queues/schedulers, RPC/streaming, and multi-process/multi-threaded runtime behavior.
• Substantial open-source contributions to vLLM, SGLang, PyTorch, Triton, NCCL or related GPU/inference infrastructure; maintainer experience preferred.