Senior Software Engineer, CoreAI Workload Engines | Microsoft Careers

What you'll do

• Optimize inference engines for OpenAI and open-source models by implementing performance improvements.
• Run end-to-end experiments to measure and improve latency, throughput, availability, and cost of AI workloads.
• Build experimentation capabilities for large-scale AI inference to ensure rapid, safe iteration.
• Own serving availability and efficiency for Azure OpenAI Service through tiered experimentation and multi-modal utilization.
• Design and evolve inference serving architectures using techniques like disaggregated serving and quantization.
• Extend AI infrastructure abstractions to support elastic, heterogeneous inference engines at scale.

What we're looking for

• Proven ability to design and operate large-scale production inference services.
• Strong skills in performance analysis including benchmarking, profiling, diagnosing regressions.
• Hands-on experience with Kubernetes for building and operating services.
• Demonstrated technical leadership and cross-team architectural alignment.
• Experience optimizing LLM inference in production environments (preferred).
• Familiarity with GPU-accelerated inference stacks and high-performance networking.