AI/ML Evaluation Specialist, Human Data

Apple Inc

Confirmed live today High trust

Quick summary

Work type: On-site
Location: Cupertino, CA
Salary: $144,600–$263,800 / yr
Posted: 117 days ago
Freshness: Confirmed live today
Nearby: 99+ roles within 25 mi

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $221k

This role $204k

$129k $294k

below market most similar roles pay here above market

This role pays less than 62% of similar roles. Most pay $187,850–$254,750 — the blue band above. At the midpoint, this role pays about $204k versus about $221k for comparable roles.

Based on 240 similar postings.

Employer

About Apple Inc

Apple Inc. is a multinational technology company known for designing and manufacturing consumer electronics, software, and online services, including the iPhone, Mac, iPad, and App Store. Industry: Consumer Electronics & Software

Apple Inc currently has 3497 open roles on FindRole.

Listed pay typically runs $174,149–$277,600 across 2702 roles with salary data.

Most-posted roles

View all roles at Apple Inc

At a glance

TL;DR · AI/ML Evaluation Specialist, Human Data

Role Posting Log in to save

The AI/ML Evaluation Specialist, Human Data joins the Human-centered AI team within Apple Services Engineering to lead complex, multi-stakeholder operations for data collection, curation, and human evaluation. This role involves owning the operational strategy for large-scale, multilingual human data programs across Apple Music, App Store, TV+, Podcasts, and Books. You will design onboarding scaffolds, analyze annotator behavior, and build automation and feedback systems to improve data quality for generative AI features. Key responsibilities include managing data pipelines, enforcing quality frameworks, and ensuring compliance with privacy and legal standards. The role requires proficiency in Python, R, and SQL to process large datasets. Candidates must possess expertise in NLP/NLU, human-in-the-loop data collection, and statistical process controls to solve problems related to human alignment and data diversity.

Skills

Python R SQL ETL NLP NLU Generative AI Human-in-the-loop Data Pipelines Data Annotation Statistical Process Controls Machine Learning Data Quality Frameworks AI Safety Responsible AI

What you'll do

Lead end-to-end execution of human data collection programs for multilingual and multimodal AI features.
Design and own measurement frameworks to track spend, speed, inter-rater reliability, and volume.
Build and implement quality frameworks using statistical process controls to detect and remediate data degradation.
Analyze workflows to identify inefficiencies and implement automated agent-in-the-loop and machine-in-the-loop pipelines.
Design compliant data collection and user study programs in partnership with Legal, Privacy, and Security teams.
Manage the data collection lifecycle for internal and external workforces, including onboarding and calibration programs.
Translate human-centered AI principles into scalable task designs that reduce cognitive burden for annotators.

What we're looking for

Bachelor's degree or higher in Cognitive Science, Linguistics, or a related field with an experimental or empirical component.
Master's degree or higher in Cognitive Science, Linguistics, or a related field with an experimental or empirical component (preferred).
4+ years of experience leading cross-team human data programs for AI/ML in an NLP/NLU or generative AI environment.
2+ years of experience owning data strategy for frontier AI development and human alignment methodologies (preferred).
Proficiency in programming and data languages including Python, R, and SQL.
Hands-on experience managing 0→1 human-in-the-loop data collection, annotation, and evaluation initiatives.
Expertise in end-to-end data annotation quality management, including statistical process controls and data quality metrics.
Experience working cross-functionally with engineering, data science, legal, privacy, and third-party suppliers.

Save