As a senior AI systems architect at our cutting-edge team, you will lead the co-design of next-generation AI systems across hardware and software boundaries, focusing on accelerators, interconnects, memory systems, storage, runtimes, and distributed training frameworks. Your daily tasks include analyzing real workloads to identify bottlenecks and translating insights into actionable system requirements, optimizing parallelism strategies for large-scale AI platforms, and developing performance models to guide future hardware roadmaps. You will collaborate closely with compiler, kernel, and runtime teams to unlock the full potential of current and next-generation accelerators, influence AI hardware design at both system and silicon levels, and mentor senior engineers while setting technical direction across the organization. Proficiency in C/C++, CUDA, Python, and experience with distributed systems, high-performance computing, and ML systems is essential, as well as a deep understanding of large-scale AI clusters and LLMs.
Skills
Python
C/C++
CUDA
Distributed Systems
HPC
ML Systems
Runtimes
Compilers
Performance Modeling
Benchmarking
Systems Analysis
Hardware-Silicon Co-Design
AI Accelerators
GPU Architectures
NCCL
MPI
RDMA
InfiniBand
CI/CD
What you'll do
Lead the co-design of AI systems across hardware and software boundaries.
Analyze real workloads to identify bottlenecks and translate findings into system requirements.
Optimize parallelism strategies and distributed algorithms for large-scale AI systems.
Develop performance models to guide future hardware and platform roadmaps.
Partner with teams to unlock full performance of current and next-gen accelerators.
Influence AI hardware design at system and silicon levels, including microarchitecture.
Lead cross-functional efforts to prototype and productionize high-impact co-design ideas.
What we're looking for
Strong background in AI accelerator or GPU architectures, distributed systems, and large-scale AI training/inference.
Proficiency in systems-level programming (C/C++, CUDA, Python) and performance-critical software development.
Experience designing or operating large-scale AI clusters for training or inference.
Deep familiarity with LLMs, multimodal models, recommendation systems, and their system implications.
Expertise in accelerator interconnects, communication stacks, and hardware–software co-design for AI workloads.
Proven ability to influence technical decisions across multiple organizational boundaries.
Background in performance modeling, capacity planning, and leading hardware roadmaps.
Market check
Salary context
This
$139,900–$274,800
range sits above
60%
of similar postings on FindRole.
Peer median band
$144,600–$257,050
Median floor and ceiling across peers.
Typical midpoint (25–75%)
$177,250–$235,750
Middle half of comparable postings.
Based on 240 comparable postings.*
* 240 is the maximum number of comparable postings sampled.
Employer
About Microsoft
Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing
Microsoft currently has
534 open roles
on FindRole.
Listed pay typically runs
$119,800–$234,700
across 488 roles with salary data.