As a Principal Software Engineer on Microsoft’s AI Frameworks team, you will work across multiple layers of the AI software stack, including abstractions, programming models, compilers, runtimes, libraries, and APIs, to enable large-scale training and inference of AI models. Your daily tasks will involve benchmarking OpenAI and other LLMs for performance on GPUs and Microsoft hardware, debugging and optimizing their performance, and monitoring regressions to ensure continuous improvements in deployment speed and efficiency. You will collaborate closely with researchers and engineers to deliver scalable, production-ready solutions that support major Microsoft products like Office, Windows, Bing, SQL Server, and Dynamics. The role requires expertise in C++ or Python, experience with high-performance applications on CPUs/GPUs, familiarity with DL frameworks such as PyTorch, TensorFlow, or ONNX Runtime, and proficiency in GPU profiling tools.
Skills
Python
C++
PyTorch
TensorFlow
ONNX_Runtime
CUDA
ROCm
Triton
GPU_Profiling_Tools
Docker
CI/CD
Linux
Windows
What you'll do
Benchmark OpenAI and other LLM models for performance on GPUs and Microsoft hardware.
Debug and optimize training/inference workloads on CPUs/GPUs for better performance.
Monitor performance regressions and drive continuous improvements in deployment efficiency.
Enable large-scale AI model deployments with minimal hardware requirements.
Collaborate with researchers to deliver scalable, production-ready AI performance enhancements.
What we're looking for
Master's Degree in Computer Science or related field with 12+ years experience, or Bachelor's with 15+ years experience in technical engineering.
Proficient in coding languages such as C++ or Python for high performance applications and debugging on CPUs/GPUs.
Experience working with DNN/LLM inference and familiarity with DL frameworks like PyTorch, TensorFlow, ONNX Runtime, CUDA, ROCm, Triton.
Strong background in software engineering principles, computer architecture, GPU architecture, and hardware neural net acceleration.
Expertise in end-to-end performance analysis and optimization of state-of-the-art LLMs and HPC applications using GPU profiling tools.
Ability to independently lead projects and collaborate effectively with cross-functional teams.
Market check
Salary context
Above market
How this pay compares to similar roles
Similar $196k
This role $209k
$127k
most similar roles pay here
$291k
This role pays more than
69%
of similar roles. Most pay
$177,250–$214,500
— the shaded band above.
At the midpoint, this role pays about
$209k
versus about
$196k
for comparable roles.
Based on 240 similar postings.
Employer
About Microsoft
Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing
Microsoft currently has
534 open roles
on FindRole.
Listed pay typically runs
$119,800–$234,700
across 488 roles with salary data.