below marketmost similar roles pay hereabove market
This role pays more than
64%
of similar roles. Most pay
$144,350–$202,120
— the blue band above.
At the midpoint, this role pays about
$177k
versus about
$173k
for comparable roles.
Based on 240 similar postings.
Employer
About Microsoft
Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing
Microsoft currently has
651 open roles
on FindRole.
Listed pay typically runs
$119,800–$234,700
across 605 roles with salary data.
The Senior Software Engineer - AI Infrastructure joins the AI Infrastructure team to build the AI data-plane powering LLM inferencing workloads. In this role, you will design and implement scalable distributed systems for Azure AI traffic management, scheduling, and workload optimization. You will develop algorithms for intelligent workload distribution across global infrastructure while meeting latency and cost objectives. Key responsibilities include building observability tooling, leading production issue investigations, and mentoring engineers. The role requires proficiency in languages such as C, C++, C#, Java, JavaScript, or Python. Candidates should possess experience with telemetry, resource scheduling, container orchestration like Kubernetes, and distributed storage systems. This work focuses on the technical challenge of serving large-scale models reliably and efficiently across premium AI infrastructure.
What does a Software Engineer earn?
Median $196000 from 2512 postings across 144 companies.
Design and implement scalable distributed systems for Azure AI traffic management and scheduling.
Drive technical direction for routing, capacity allocation, and workload optimization services.
Develop algorithms to distribute AI workloads across global infrastructure while meeting latency and cost objectives.
Build observability and operational tooling to monitor service health and traffic patterns in real-time.
Lead investigations into complex production issues to improve reliability and platform automation.
Identify opportunities to improve fleet efficiency and optimize the utilization of premium AI infrastructure.
Mentor engineers and review designs and code to raise engineering standards across the organization.
What we're looking for
Bachelor's Degree in Computer Science or related technical field and 4+ years of technical engineering experience with coding in languages like C, C++, C#, Java, JavaScript, or Python.
Master's Degree in Computer Science or related technical field and 6+ years of technical engineering experience with coding in languages like C, C++, C#, Java, JavaScript, or Python (preferred).
Bachelor's Degree in Computer Science or related technical field and 8+ years of technical engineering experience with coding in languages like C, C++, C#, Java, JavaScript, or Python (preferred).
Experience with telemetry, metrics pipelines, or resource scheduling systems (preferred).
Familiarity with cloud platforms (Azure, AWS, GCP) and container orchestration (Kubernetes, Service Fabric) (preferred).
Experience operating large-scale services with stringent reliability and availability requirements (preferred).
Experience with distributed storage systems, telemetry platforms, and real-time data processing (preferred).
Familiarity with AI/ML infrastructure, GPU fleets, inference serving systems, or large-scale model deployment platforms (preferred).
Ability to pass the Microsoft Cloud Background Check and other security screening requirements.
Python
C/C++
Large Language Models
AI Agents
Retrieval-Augmented Generation
CUDA
MLOps
GPU Computing
Data Pipelines
APIs
system-design
Production Software
Model Evaluation
AI Observability
Workflow Orchestration
Silicon Bring-up
Distributed Systems
LLM Gateway
Model Serving
Inference
Feature Stores
Training Pipelines
Observability
System Design
Latency Optimization
Scalability
Open-source Infrastructure