Principal Software Engineer, CoreAI Workload Engines

What you'll do

• Optimize inference engines for OpenAI and open-source models by implementing performance improvements.
• Build and use experimentation capabilities for large-scale AI inference to ensure quick and safe iterations.
• Own serving availability and efficiency for Azure OpenAI Service workloads through tiered experimentation and multi-modal utilization.
• Design and evolve inference serving architectures to improve utilization and latency using advanced techniques like disaggregated serving.
• Extend AI infrastructure abstractions to support elastic, heterogeneous inference engines reliably at scale.

What we're looking for

• Proven ability to design and operate large-scale production inference services.
• Strong skills in performance analysis including benchmarking, profiling, diagnosing regressions.
• Hands-on experience with Kubernetes for building and operating services.
• Demonstrated technical leadership and cross-team architectural alignment.
• Experience optimizing LLM inference in production environments.
• Familiarity with GPU-accelerated inference stacks and high-performance networking.