AI Experience Researcher, Product Evaluation, Vision Products Group

What you'll do

• Develop scalable automated evaluation methodologies for AI experiences.
• Produce comprehensive evaluation plans including scope, validation, data strategy, and timelines.
• Derive experimental designs and write test instructions for LLM judges or human raters.
• Define requirements and curate datasets representing realistic usage scenarios.
• Implement and analyze automated evaluations to identify key insights and areas for improvement.
• Prepare and present clear evaluation findings with actionable recommendations for product decisions.

What we're looking for

• Advanced degree in Cognitive Psychology, HCI, UX Research, Learning Sciences, or related field focusing on human cognition and empirical evaluation
• Strong data-driven mindset with experience designing rigorous empirical research and evaluating complex human-system interactions
• Ability to translate theoretical knowledge into practical evaluation frameworks and measurement designs for AI experiences
• Proficiency in data analysis and statistical validity, with a track record of actionable insights from qualitative and quantitative data
• Exceptional collaboration skills working effectively in cross-functional teams including engineering, ML, design, QA, and leadership
• Strong communication skills to translate complex research findings into clear recommendations for both technical and non-technical audiences