| Microsoft Careers

Microsoft

Quick summary

Work type: On-site
Location: —
Salary: $119,800–$234,700 / yr
Posted: 88 days ago
Closes: Sep 19, 2026

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $186k

This role $177k

$106k most similar roles pay here $248k

This role pays more than 62% of similar roles. Most pay $163,500–$208,800 — the shaded band above. At the midpoint, this role pays about $177k versus about $186k for comparable roles.

Based on 240 similar postings.

Employer

About Microsoft

Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing

Microsoft currently has 1633 open roles on FindRole.

Listed pay typically runs $119,800–$234,700 across 1454 roles with salary data.

Most-posted roles

View all roles at Microsoft

At a glance

TL;DR · | Microsoft Careers

Apply Now Log in to save

As a Senior Software Engineer in the CoreAI team at Microsoft Azure, you will play a pivotal role in developing the AI data-plane that powers large language model (LLM) inferencing workloads for diverse clients. Your responsibilities include designing and implementing core serving systems, smart routing, and request distribution for various LLMs, ensuring reliability, scalability, and ultra-low latency. You will leverage C++, Java, or similar languages to build large-scale AI services and platform capabilities that drive new product experiences. This role requires expertise in distributed systems, system performance optimization, and modern AI-assisted development tools, with a focus on delivering high-quality solutions at scale for Microsoft’s expansive catalog of LLMs.

Skills

C++ Java Python Docker Kubernetes Terraform AWS Azure CI/CD Prometheus Grafana PostgreSQL Redis MongoDB Distributed Systems Microservices RESTful APIs GraphQL Observability Scalability Reliability

What you'll do

Design and implement core serving systems for large language models (LLMs).
Develop smart routing and request distribution mechanisms for diverse LLMs.
Build scalable AI services to power new products and customer experiences.
Drive innovation in AI system architecture and engineering best practices.
Enhance reliability, scalability, observability of mission-critical AI services.
Mentor team members through code reviews and technical guidance.

What we're looking for

4+ years of experience in designing and coding large-scale AI services and platforms.
Proficient in C, C++, C#, or Java for developing core serving systems and smart routing.
Expertise in building high-quality, reliable systems with a focus on scalability and performance.
Understanding of distributed systems and request serving at scale across global infrastructure.
Experience using modern AI-assisted development tools to enhance engineering efficiency and quality.
Hands-on technical leadership in architecture design, code reviews, and mentorship.

Save