Distinguished Engineer, AIML Foundation Model

What you'll do

• Define the technical vision and roadmap for foundation model inference engines and supporting systems.
• Optimize inference performance through techniques like quantization, speculative decoding, and hardware-aware optimizations.
• Architect inference systems that support diverse workloads including data generation, large-scale evaluation, and LLM-as-judge scoring.
• Expand infrastructure capabilities into adjacent areas such as training pipelines, data systems, and evaluation harnesses.
• Translate requirements from modeling teams into a coherent platform with clear interfaces and prioritized milestones.
• Collaborate with researchers to integrate state-of-the-art techniques into the production development loop.
• Lead and mentor a diverse team of engineers while resolving technical trade-offs and ensuring project delivery.

What we're looking for

• MS or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent industry experience.
• 15+ years of experience building large-scale ML or distributed systems with a track record of technical leadership and impact.
• Deep hands-on expertise in foundation model inference engines to improve performance, efficiency, and reliability at scale.
• Experience supporting diverse foundation model inference use cases with varying throughput, latency, cost, and quality constraints.
• Ability to contribute to adjacent systems areas including training infrastructure, data systems, or evaluation.
• Deep understanding of GPU/TPU/accelerator architecture, distributed systems, and model optimization techniques like quantization and compilation.
• Proficiency with ML frameworks such as JAX and PyTorch, along with inference and serving stacks.
• Proven experience leading engineers, mentoring staff, and partnering with research teams to productionize models (preferred).