Machine Learning Engineer, ML/GenAI Evaluation

What you'll do

• Define evaluation criteria and quality metrics for ML models powering Wallet features
• Design adversarial test strategies to surface model failure modes before they reach users
• Develop robustness testing methodologies including distribution shift and out-of-distribution generalization
• Own end-to-end fairness evaluation, defining metrics and building bias test suites
• Synthesize evaluation results into clear insights guiding model development priorities
• Establish and maintain user persona–stratified benchmarks reflecting global user diversity
• Own the final quality sign-off process for models before they ship to users

What we're looking for

• M.S. in Machine Learning, Computer Science, Statistics, Applied Mathematics, or related field preferred; 7+ years hands-on ML experience required.
• Deep expertise in model evaluation, offline metrics design, and behavioral testing for production systems.
• Strong track record designing evaluation frameworks beyond standard accuracy/F1 metrics to include fairness, precision-recall tradeoffs, calibration.
• Proven ability to construct adversarial test suites and edge-case corpora to identify failure modes before deployment.
• Experience with Python, evaluation tooling, data pipelines, and experiment tracking (e.g., MLflow).
• Excellent communication skills to translate metric results into product-quality narratives for diverse audiences.