This role pays more than
67%
of similar roles. Most pay
$151,000–$202,120
— the shaded band above.
At the midpoint, this role pays about
$177k
versus about
$177k
for comparable roles.
Based on 240 similar postings.
Employer
About Microsoft
Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing
Microsoft currently has
518 open roles
on FindRole.
Listed pay typically runs
$119,800–$234,700
across 505 roles with salary data.
As a Senior Software Engineer - Performance Tooling on the AI Frameworks team, you will work across multiple layers of the AI software stack, including abstractions, programming models, compilers, runtimes, libraries, and APIs. You will be responsible for benchmarking OpenAI and other large language models for performance on GPUs and Microsoft hardware while debugging, profiling, and optimizing training and inference workloads on CPUs and GPUs. Your daily work involves monitoring performance regressions to reduce deployment time and hardware footprints. To succeed, you must possess proficiency in C++ or Python, along with experience in deep learning frameworks like PyTorch, TensorFlow, or ONNX Runtime. You will also utilize CUDA, ROCm, and Triton while applying knowledge of computer architecture and GPU neural net acceleration to solve complex performance challenges for large-scale AI models.
What does a Software Engineer earn?
Median $195000 from 1914 postings across 140 companies.
Develop software across multiple layers of the AI stack including compilers, runtimes, and libraries.
Benchmark OpenAI and other large language models on GPUs and Microsoft hardware.
Debug, profile, and optimize performance for training and inference workloads on CPUs and GPUs.
Monitor performance regressions to drive continuous improvements in deployment speed and hardware efficiency.
Reduce the hardware footprint required to deploy state-of-the-art AI models.
Deliver scalable, production-ready AI performance improvements in collaboration with researchers and engineers.
What we're looking for
Bachelor's Degree in Computer Science or related technical field and 4+ years of experience coding in C++, Python, or equivalent.
Master's Degree in Computer Science or related field and 6+ years of engineering experience (preferred).
Bachelor's Degree in Computer Science or related field and 8+ years of engineering experience (preferred).
Ability to pass Microsoft Cloud background checks and government security screening requirements.
4+ years of practical experience in high performance applications and performance debugging/optimization on CPUs and GPUs (preferred).
Experience in DNN/LLM inference, deep learning frameworks (PyTorch, TensorFlow, or ONNX Runtime), and familiarity with CUDA, ROCm, or Triton (preferred).
Technical background in software engineering principles, computer architecture, GPU architecture, and hardware neural net acceleration (preferred).
Experience in end-to-end performance analysis of LLMs/HPC applications using GPU profiling tools (preferred).
Python
C++
PyTorch
ONNX Runtime
vLLM
TensorRT-LLM
AI Model Architectures
Inference Systems
Distributed Systems
Hardware-Software Debugging
Performance Analysis
Accelerator Architectures
Automation