Principal Software Engineer | Microsoft Careers

What you'll do

• Optimize large-scale deep learning inference for Microsoft’s advertising platforms.
• Accelerate GPU inference performance across various AI-native surfaces like Copilot and Search.
• Bridge advanced GPU technologies with critical business applications in the monetization platform.
• Identify and resolve CPU/GPU bottlenecks using profiling tools such as Nsight and TensorBoard.
• Build high-throughput inference serving stacks for continuous batching and KV-cache optimizations.
• Enhance model efficiency through techniques like quantization, distillation, and low-rank methods.

What we're looking for

• Solid experience in GPU inference optimization using CUDA, TensorRT, or custom GPU kernels.
• Proficiency with profiling tools like Nsight and TensorBoard for identifying CPU/GPU bottlenecks.
• Deep understanding of LLM/SLM architectures including attention mechanisms, embeddings, MoE, and decoders.
• Experience optimizing latency-critical online services for real-time performance.
• Familiarity with model compression techniques such as quantization, distillation, SVD, and low-rank methods.
• Expertise in building high-throughput inference serving stacks with continuous batching and KV-cache optimizations.