This role pays more than
55%
of similar roles. Most pay
$170,000–$256,587
— the shaded band above.
At the midpoint, this role pays about
$209k
versus about
$213k
for comparable roles.
Based on 240 similar postings.
Employer
About Microsoft
Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing
Microsoft currently has
598 open roles
on FindRole.
Listed pay typically runs
$119,800–$234,700
across 586 roles with salary data.
As a Principal Applied Scientist for Copilot Evals on the CADET team, you will establish the scientific direction for customer-grounded quality systems. You will work directly with enterprise customers and internal teams to translate high-value workflows, recurring pain points, and user research into trusted evaluation signals and post-training priorities. Your daily responsibilities include defining what constitutes "good" across various quality attributes, designing behavior rubrics, and developing methods to assess evaluation set representativeness and grader reliability. You will build reusable assets such as RLEs, reward signals, and training data while diagnosing model failures in grounding, retrieval, and orchestration. The role requires expertise in machine learning, natural language processing, information retrieval, reinforcement learning, and statistical inference. You will solve complex problems regarding how to translate qualitative customer expectations into measurable, scalable metrics for agentic systems and large language models.
What does a Applied Scientist earn?
Median $208800 from 47 postings across 10 companies.
Define the scientific strategy for customer-grounded quality across priority Copilot intents and safety attributes.
Translate user research and production incidents into evaluation priorities and post-training assets.
Develop methods to assess evaluation set representativeness, coverage, grader reliability, and alignment with production outcomes.
Design behavior evaluations, rubrics, and calibration methods to turn qualitative expectations into measurable metrics.
Attribute quality losses across grounding, retrieval, reasoning, and generation to link offline results with online signals.
Mentor scientists and engineers while influencing evaluation and post-training strategy across organizational boundaries.
What we're looking for
Bachelor's degree plus 6 years of experience, Master's plus 4 years, or Doctorate plus 3 years in a quantitative field.
Equivalent practical experience to the specified educational and experience requirements.
Significant experience applying machine learning, NLP, information retrieval, reinforcement learning, or evaluation methods to production systems (preferred).
Experience designing evaluations, metrics, experiments, datasets, graders, or reward functions for AI or agentic systems (preferred).
Ability to translate qualitative judgments into testable hypotheses and measurable evaluation criteria (preferred).
Solid understanding of statistical inference, sampling, measurement validity, bias, uncertainty, and experimental design (preferred).
Experience with LLMs, copilots, agents, tool use, RAG, or enterprise grounding (preferred).
Experience running or partnering on RLHF, DPO, instruction tuning, fine-tuning, or human-preference data programs (preferred).
Large Language Models
Information Retrieval
Ranking Systems
Agentic AI
Retrieval-Augmented Generation
Supervised Fine-Tuning
Preference Optimization
Model Distillation
Machine Learning
A/B Testing
Distributed Training
Search Engines
Machine Learning
Large Language Models (LLMs)
Deep Learning
Azure
AI Safety
Supervised Fine-Tuning
Distillation
Policy Optimization
Distributed Systems
Algorithms
Software Engineering
Predictive Analytics
Statistics
Data Science
Machine Learning
Anomaly Detection
Large Language Models
Deep Learning
Causal Inference
Statistical Machine Learning
Data Mining
System Design
Low-latency Pipelines
Experimentation Frameworks
Model Operationalization