Machine Learning Engineer, ML/GenAI Evaluation

What you'll do

• Define evaluation criteria and quality metrics for ML models powering Wallet features
• Design adversarial test strategies to surface model failure modes before they reach users
• Develop robustness testing methodologies including distribution shift and out-of-distribution generalization
• Own end-to-end fairness evaluation, defining metrics and building bias test suites across user populations
• Synthesize evaluation results into actionable insights guiding model development priorities and product decisions
• Establish and maintain structured test sets reflecting the diversity of Apple's global user base

What we're looking for

• M.S. in Machine Learning, Computer Science, Statistics, Applied Mathematics, or related field preferred; 7+ years hands-on ML experience required.
• Deep expertise in model evaluation, offline metrics design, and behavioral testing for production systems.
• Strong track record designing evaluation frameworks beyond standard accuracy/F1 metrics to include fairness, calibration, and task-specific quality dimensions.
• Proven ability to construct adversarial test suites and edge-case corpora that surface model failure modes before deployment.
• Experience with Python programming, evaluation tooling, data pipelines, and experiment tracking (e.g., MLflow).
• Excellent communication skills for translating metric results into product-quality narratives for diverse audiences.