Principal Applied Scientist (CoreAI) | Microsoft Careers

Microsoft

Quick summary

Work type
On-site
Location
WA
Salary
$165,600–$296,400 / yr
Posted
15 days ago
Closes
Nov 17, 2026

Market check

Salary context

Above market

How this pay compares to similar roles

Similar $193k
This role $231k
$121k most similar roles pay here $315k

This role pays more than 98% of similar roles. Most pay $177,250–$208,800 — the shaded band above. At the midpoint, this role pays about $231k versus about $193k for comparable roles.

Based on 239 similar postings.

Employer

About Microsoft

Microsoft Corporation is a global technology leader producing software, hardware, and cloud services including Windows, Office 365, Azure cloud platform, Xbox gaming, and Surface devices. Industry: Software & Cloud Computing

Microsoft currently has 728 open roles on FindRole.

Listed pay typically runs $119,800–$234,700 across 664 roles with salary data.

Most-posted roles

View all roles at Microsoft

At a glance

TL;DR · Principal Applied Scientist (CoreAI) | Microsoft Careers

Apply Now Log in to save

As an Applied Sciences IC6 at Core AI, you will join a pioneering team shaping the future of software development through Generative AI. Your primary responsibility is to develop and apply scientific methods for evaluating and measuring single-agent and multi-agent systems in production environments, focusing on quality, safety, reliability, cost, and behavioral consistency. You will design evaluation frameworks that integrate offline evaluations with real-time telemetry data to assess how changes impact agent performance, define quality benchmarks, build models for anomaly detection, and collaborate with engineering teams to operationalize these methods in production. Key technologies include Python, LangChain, OpenAI SDK, Azure Monitor, and distributed tracing systems. This role demands expertise in applied machine learning, statistical analysis, and observability data handling, as well as a background in AI safety and responsible AI practices.

Skills

Python LangChain Azure Monitor OpenAI SDK Distributed Tracing Causal Inference Statistical Methods Machine Learning Evaluation Frameworks Observability Systems Responsible AI LLM Systems LangGraph Application Insights Advanced Degree required_experience_applied_science microsoft_cloud_scale_monitoring

What you'll do

  • Develop evaluation frameworks for single-agent and multi-agent systems to measure quality, safety, reliability, cost, and behavioral consistency.
  • Design methodologies linking offline evaluations, online signals, and production telemetry to assess real-world agent performance impacts.
  • Define scientifically grounded quality benchmarks for AI agents, including task success, tool-use effectiveness, plan quality, and user outcomes.
  • Build models and analysis techniques to detect regressions, identify root causes, and characterize diverse agent behaviors across environments.
  • Advance observability through new approaches like trace analysis, health modeling, behavioral clustering, anomaly detection, and multi-agent coordination.

What we're looking for

  • 6+ years of experience in applied science, machine learning, or related technical fields.
  • Strong background in designing evaluation methodologies and measurement systems for complex intelligent systems.
  • Experience analyzing large-scale production data to drive product improvements.
  • Proficient coding skills in Python with a focus on working with engineering teams on production systems.
  • Advanced degree in Computer Science, Machine Learning, Statistics, Applied Mathematics, or related field.
  • Familiarity with agent frameworks and evaluation frameworks for AI systems.
  • Background in AI safety, guardrails, and responsible AI measurement.

Related searches

More Principal Applied Scientist jobs All jobs at Microsoft

More like this

Similar roles

Senior and Principal Applied Scientists - CoreAI | Microsoft Careers

Microsoft

US 42 days ago $119,800–$234,700
Python Azure Monitor Application Insights LangChain OpenAI SDK Distributed Tracing Causal Inference Statistical Methods Machine Learning LLM-based Systems Observability Data Evaluation Frameworks Responsible AI A/B Testing Anomaly Detection Behavioral Analysis Agent Orchestration Cloud-Scale Monitoring Large-Scale Production Data
Save

Principal Software Engineer, CoreAI | Microsoft Careers

Microsoft

US 85 days ago $139,900–$274,800
Python C++ Java JavaScript Azure CI/CD Kubernetes Docker Terraform Prometheus Grafana LLMs SLMs Multimodal_Models Code_Specific_Models Scalability Reliability Security Privacy Cloud_Infrastructure DevOps
Save

| Microsoft Careers

Microsoft

US 80 days ago $142,800–$274,800
Kubernetes Python C C++ Java JavaScript Terraform AWS Azure PostgreSQL CI/CD Prometheus Grafana Docker RDMA InfiniBand NCCL CUDA AKS Dynamic Resource Allocation(DRA)
Save