As a Principal Engineer on the CoreAI GPU Infrastructure team, you will design and build foundational accelerated compute platforms for large-scale AI training and inference across Azure. Your daily tasks include developing systems for GPU device management, scheduling, isolation, and sharing, as well as building advanced orchestration capabilities using Kubernetes and related ecosystems to ensure efficient resource utilization. You will also optimize performance and reliability across extensive GPU fleets and collaborate with networking teams to enable high-performance interconnects. This role requires expertise in C++, Java, Python, and other languages, along with experience in virtualization and container platforms like Kubernetes. Ideal candidates have a background in large-scale infrastructure design and operation, strong problem-solving skills, and the ability to mentor engineers while driving technical direction across teams. The position offers significant impact on cloud GPU platform development and collaboration with experts from various domains.
Skills
Kubernetes
Python
C
C++
Java
JavaScript
Terraform
AWS
Azure
PostgreSQL
CI/CD
Prometheus
Grafana
Docker
RDMA
InfiniBand
NCCL
CUDA
AKS
Dynamic Resource Allocation(DRA)
What you'll do
Design and build GPU-accelerated infrastructure for diverse environments.
Develop systems for efficient GPU device management and multitenant usage.
Build advanced orchestration scenarios using Kubernetes ecosystem capabilities.
Optimize performance and utilization across large-scale GPU fleets.
Partner with networking teams to enable high-performance interconnects.
Drive end-to-end platform features from design through production deployment.
Influence platform architecture and technical direction through leadership.
What we're looking for
Bachelor's Degree in Computer Science or related field with 6+ years of technical engineering experience.
Proven ability to design and operate large-scale production infrastructure with high reliability and performance.
Strong problem-solving skills for debugging complex cross-layer systems issues.
Hands-on experience with virtualization and container platforms like VMs, Kubernetes, and container runtimes.
Demonstrated technical leadership in mentoring engineers and driving architectural alignment across teams.
Experience in building or operating multitenant AI platforms in cloud environments.
Market check
Salary context
This
$142,800–$274,800
range sits above
70%
of similar postings on FindRole.
Peer median band
$139,900–$245,600
Median floor and ceiling across peers.
Typical midpoint (25–75%)
$177,250–$214,500
Middle half of comparable postings.
Based on 240 comparable postings.*
* 240 is the maximum number of comparable postings sampled.
Employer
About Microsoft
Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing
Microsoft currently has
534 open roles
on FindRole.
Listed pay typically runs
$119,800–$234,700
across 488 roles with salary data.