below marketmost similar roles pay hereabove market
This listing doesn't post a salary. Most similar roles pay
$173,285–$226,350.
Based on 240 similar postings.
Employer
About Apple Inc
Apple Inc. is a multinational technology company known for designing and manufacturing consumer electronics, software, and online services, including the iPhone, Mac, iPad, and App Store. Industry: Consumer Electronics & Software
Apple Inc currently has
3497 open roles
on FindRole.
Listed pay typically runs
$174,149–$277,600
across 2702 roles with salary data.
The Senior Manager, Evaluation, Services Special Projects leads a team of applied scientists and machine learning engineers to measure the quality of a new conversational experience. This role involves building and owning specific components of an evaluation plan, including methods, datasets, and automated grading systems that translate product behavior into quality signals. You will set the technical direction for large language model based autograders, rubrics, and evaluation set development while pairing these with logging, instrumentation, and online experimentation. Key responsibilities include mentoring staff, recruiting talent, and partnering with engineering and quality assurance teams to diagnose failures. The work focuses on evaluating request interpretation, retrieval, ranking, groundedness, and task execution through external integrations, ensuring the system is correct, relevant, and trustworthy before shipping.
Build and lead a team of applied scientists and machine learning engineers to measure product quality.
Design and run evaluation methods for conversational AI, including request interpretation, retrieval, ranking, and response composition.
Set the technical direction for LLM-based autograders, including rubric design, evaluation set development, and calibration.
Develop automated grading systems and metrics that translate product behavior into clear quality signals for leadership.
Pair automated evaluation with logging, instrumentation, and online experimentation to measure measurable quality gains.
Translate broader evaluation strategies into concrete methods, deliverables, and explicit quality gates for release decisions.
Review designs and experiments while staying close to data to diagnose failures and ship fixes.
Recruit, mentor, and grow a team of senior individual contributors and technical leads.
What we're looking for
6+ years of experience in machine learning, data science, or machine learning evaluation.
3+ years of direct people management of scientists or engineers.
Track record of hiring, developing, and leading teams of applied scientists, data scientists, or machine learning engineers.
Hands-on experience building and deploying LLM-based autograders and rubrics, including rubric design, evaluation set development, validation, and calibration.
Experience evaluating conversational AI, search, recommendation, or ranking systems using offline metrics and online experimentation.
Strong command of statistics and experimental design, including A/B testing and interpreting noisy metrics.
Exceptional written and verbal communication skills with a track record of influencing engineers and senior leaders.
MS or PhD in Computer Science, Machine Learning, Statistics, or a related quantitative field (preferred).
Machine Learning Evaluation
LLM
Conversational AI
A/B Testing
Quality Assurance
API Integration
Logging Infrastructure
Data Science
Search Systems
Autograders
Instrumentation
Test Engineering
Systems Engineering
Project Management
Earned Value Management
Six Sigma
CMMI
Lean
Root Cause Analysis
Aero-Mechanical
Aero-Electrical
Structural
Stress
Mass Properties
Aerodynamics
Software
MTC