| Microsoft Careers

Microsoft

Hybrid Actively hiring Posted this week
Redmond, WA Posted 4 days ago $142,800–$274,800 / year
View original post Log in to save

At a glance

AI generated

TL;DR

As a Principal Software Engineering Manager for the M365 Copilot inference team, you will lead a strategic initiative aimed at maximizing throughput per GPU in the Copilot inference stack. Your day-to-day responsibilities include building and leading a high-performing engineering team focused on optimizing model execution and runtime performance, defining strategies to enhance efficiency, and partnering with cross-functional teams to co-design and productionize advanced optimizations. You will establish metrics and frameworks to measure performance gains, ensure live-site operational excellence, and drive alignment across partner teams on optimization priorities. The role requires experience in leading engineering teams building backend or distributed systems, hands-on expertise in improving system throughput and resource utilization, familiarity with AI/ML inference systems, and the ability to translate technical insights into clear execution plans. This position is integral to advancing Microsoft’s applied AI capabilities at massive scale across global datacenters.

Skills

Python C++ CUDA Docker Kubernetes AWS Azure CI/CD Prometheus Grafana PostgreSQL M365 LLM_API GPU_optimization Inference_engine Large_scale_infrastructure Metrics_telemetry Experimentation_frameworks

What you'll do

  • Lead a high-performing engineering team focused on optimizing inference runtime efficiency.
  • Drive strategy to enhance throughput per GPU through advanced runtime optimizations.
  • Enable faster experimentation and iteration for rapid performance improvement rollouts.
  • Partner with cross-functional teams to co-design and implement advanced inference optimizations.
  • Establish metrics and frameworks to measure efficiency gains and guide investment decisions.

What we're looking for

  • Extensive experience leading high-performing engineering teams focused on backend or distributed systems.
  • Proven track record of improving system throughput, performance, and resource utilization in large-scale infrastructure.
  • Strong systems thinking ability to identify and optimize bottlenecks across execution, scaling, and resource management.
  • Experience driving system-level improvements in workload execution, scheduling, batching, or infrastructure efficiency.
  • Hands-on experience with developing AI/ML inference systems or GPU-based workloads.

Employer

About Microsoft

Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing

Microsoft currently has 534 open roles on FindRole.

Listed pay typically runs $119,800–$234,700 across 488 roles with salary data.

Most-posted roles

View all roles at Microsoft