Principal Applied Scientist for Copilot Evals

Microsoft

Confirmed live today High trust

Hybrid

Quick summary

Work type: Hybrid
Location: —
Salary: $142,800–$274,800 / yr
Posted: 21 days ago
Freshness: Confirmed live today
Closes: Apr 3, 2027

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $217k

This role $209k

$127k $291k

below market most similar roles pay here above market

This role pays more than 55% of similar roles. Most pay $177,250–$256,000 — the blue band above. At the midpoint, this role pays about $209k versus about $217k for comparable roles.

Based on 240 similar postings.

Employer

About Microsoft

Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing

Microsoft currently has 694 open roles on FindRole.

Listed pay typically runs $119,800–$234,700 across 651 roles with salary data.

Most-posted roles

View all roles at Microsoft

At a glance

TL;DR · Principal Applied Scientist for Copilot Evals

Role Posting Log in to save

The Principal Applied Scientist for Copilot Evals joins the CADET team to build a customer-grounded quality system for Copilot. In this role, you will work directly with enterprise customers and Copilot teams to translate high-value workflows and recurring pain points into trusted evaluation and learning signals. You will set the scientific strategy for customer-grounded quality, defining what good means and assessing whether evaluation portfolios represent real needs. Your daily work involves diagnosing model and agent failures, developing methods to assess evaluation-set representativeness, and designing behavior and task evaluations, rubrics, and graders. You will utilize skills in machine learning, natural language processing, information retrieval, reinforcement learning, and statistical inference to convert evidence into reusable RLEs, reward signals, and post-training priorities for complex production systems.

What does a Applied Scientist earn?

Median $208800 from 61 postings across 10 companies.

See salary data

Skills

Machine Learning Natural Language Processing Large Language Models Reinforcement Learning Retrieval-Augmented Generation RLHF Direct Preference Optimization Instruction Tuning Fine-tuning Statistical Inference Experimental Design Predictive Analytics Information Retrieval

What you'll do

Set the scientific strategy for customer-grounded quality and define quality tradeoffs across Copilot intents.
Translate customer feedback and production incidents into evaluation priorities and reusable post-training assets.
Develop methods to assess evaluation-set representativeness, coverage, freshness, and alignment with production outcomes.
Design behavior evaluations, rubrics, and calibration methods to turn qualitative expectations into measurable metrics.
Attribute quality losses across grounding, retrieval, tools, and model reasoning by linking offline metrics with online signals.
Maintain high standards for scientific rigor, reproducibility, and documentation while mentoring scientists and engineers.

What we're looking for

Bachelor's Degree in a quantitative field and 6+ years of related experience.
Master's Degree in a quantitative field and 4+ years of related experience.
Doctorate in a quantitative field and 3+ years of related experience.
Equivalent experience to the above educational and experience requirements.
Significant experience applying machine learning, NLP, information retrieval, or reinforcement learning to complex production systems (preferred).
Experience designing evaluations, metrics, datasets, or reward functions for AI or agentic systems (preferred).
Solid understanding of statistical inference, sampling, measurement validity, bias, uncertainty, and experimental design (preferred).
Experience with large language models, copilots, agents, RAG, or enterprise grounding (preferred).

Save