Senior Manager, Evaluation - Data Science & Insights

What you'll do

• Own the evaluation roadmap by translating organizational strategy into concrete methods and deliverables.
• Lead, recruit, and develop a team of data scientists across multiple technical disciplines.
• Execute human evaluation methodologies for large-scale consumer-facing products.
• Build and deploy LLM-based autograders and rubrics to improve model performance.
• Integrate evaluation metrics into product development cycles to drive measurable quality gains.
• Represent the team's work and results to senior leadership across AIML and SWE.
• Manage the trade-offs between feature quality and privacy for high-scale products.

What we're looking for

• Advanced degree in a quantitative field such as Statistics, Computer Science, Machine Learning, or similar.
• 6+ years of experience in data science and machine learning evaluation.
• 3+ years of experience leading technical teams.
• Demonstrated track record of running teams of 7+ data scientists and/or machine learning engineers.
• Strong experience in human evaluation methodology for consumer-facing products at scale.
• Hands-on experience building and deploying LLM-based autograders and rubrics to drive improvements in models and agentic features.
• Strong written and verbal communication skills to engage with engineers and senior leadership.
• Experience evaluating large consumer AI products, logging infrastructure, or evaluation frameworks for agentic systems (preferred).