Principal Software Engineer, LLM Optimization

What you'll do

• Own systematic benchmarking and performance characterization across all production LLM workloads.
• Design and execute quantization experiments to measure accuracy, throughput, and cost-per-token impacts.
• Drive speculative decoding strategies and own acceptance rate measurements across the model portfolio.
• Build and maintain a GPU efficiency scorecard to monitor utilization, memory headroom, and waste.
• Lead inference engine upgrade evaluations including new scheduler architectures and tensor parallelism.
• Collaborate on KV-cache optimization, prefix caching strategies, and multi-node serving architectures.
• Design and run GPU chaos engineering to test induced failure scenarios and recovery measurements.
• Architect and govern agentic AI-enabled engineering workflows to improve delivery speed and code quality.

What we're looking for

• Formal training or certification in software engineering concepts and 7+ years of applied experience.
• Deep, hands-on experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, or LLM-D.
• Strong grasp of GPU memory architecture, including KV cache sizing, memory-bandwidth vs compute bottlenecks, and quantization implications.
• Experience with quantization techniques and their real-world tradeoffs at scale.
• Familiarity with speculative decoding and the variables that drive acceptance rates in production workloads.
• Rigorous benchmarking instincts using GuideLLM, custom harnesses, or equivalent tools.
• Comfort operating in cloud GPU infrastructure at scale, specifically AWS, EKS, and managed inference services.
• Experience with disaggregated prefill/decode serving architectures, GPU hardware diagnostics, and ML observability (preferred).