This role pays more than
57%
of similar roles. Most pay
$180,375–$231,000
— the shaded band above.
At the midpoint, this role pays about
$209k
versus about
$206k
for comparable roles.
Based on 240 similar postings.
Employer
About Microsoft
Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing
Microsoft currently has
644 open roles
on FindRole.
Listed pay typically runs
$119,800–$234,700
across 624 roles with salary data.
As a Principal Software Engineer on the Copilot evaluation team, you will define the technical vision for an offline evaluation platform that ensures product quality before release. You will lead the architecture of an AI-first and agentic-first system where autonomous agents operate, extend, and heal the infrastructure. Your daily work involves building robust pipelines to evaluate multi-turn trajectories, tool selection, and end-to-end task outcomes while ensuring reliability across complex chains of dependencies under heavy load. You will develop simulation, scraping, and scoring capabilities for various modalities like voice and UX. The role requires expertise in large-scale distributed systems, reliability engineering, and AI/ML evaluation metrics. Candidates should be proficient in programming languages such as C, C++, C#, Java, JavaScript, or Python to build scalable, maintainable systems that manage the complexities of agentic behavior and automated quality gating.
What does a Software Engineer earn in Remote?
Median $202482 from 350 postings across 56 companies.
Set the technical direction for the Copilot offline evaluation platform to turn ambiguous quality questions into reproducible metrics.
Architect an AI-first, agentic-first platform where autonomous agents can operate, extend, and reason about system tools.
Ensure reliable results across complex service dependencies under heavy load using graceful degradation and intelligent retries.
Define end-to-end evaluation methods for multi-turn trajectories, tool selection, and various output modalities like voice and UX.
Build the simulation, scraping, and scoring capabilities required to measure agent behavior accurately.
Mentor engineers to build extensible systems while improving runtime cost efficiency and latency.
Integrate emerging patterns in large-scale AI evaluation and autonomous operations into the organization's standard practices.
What we're looking for
Bachelor's Degree in Computer Science or related technical field and 6+ years of engineering experience in languages like C, C++, C#, Java, JavaScript, or Python.
Ability to pass Microsoft Cloud background checks every two years.
Master's Degree in Computer Science or related field and 8+ years of engineering experience (preferred).
Bachelor's Degree in Computer Science or related field and 12+ years of engineering experience (preferred).
Deep experience with large-scale distributed systems and reliability engineering under heavy load (preferred).
Experience building AI-first or agentic systems with observability, guardrails, and safety mechanisms for autonomous agents (preferred).
Experience with AI/ML evaluation, measurement, or benchmarking systems like LLM-based grading and quality pipelines (preferred).
TypeScript
Node.js
Next.js
Prisma
Distributed Systems
relational database s
APIs
Authentication
Role-Based Access Control
Security Automation
SOC 2
NIST CSF
Web Applications
System Design