Senior Applied Scientist, Multilingual AI Evaluation

What you'll do

• Extend AI evaluation methodology and tooling to ensure features are culturally appropriate across multiple languages.
• Design and validate benchmarks and metrics that capture language-specific nuances like grammar, script, and dialect.
• Build and curate high-quality multilingual datasets and human evaluation protocols with native-speaker annotators.
• Identify systematic failure modes and capability gaps in LLMs across both high- and low-resource languages.
• Implement evaluation methods and tools using Python to ensure they run reliably at scale.
• Translate complex research findings into actionable guidance for model and product teams.
• Publish novel research regarding multilingual AI evaluation and methodology.

What we're looking for

• MS or PhD in Linguistics, Computational Linguistics, NLP, Computer Science, or a related field.
• Deep expertise in linguistics with working fluency in the structure of multiple languages beyond English.
• Strong proficiency in Python and experience with modern ML frameworks like PyTorch or JAX.
• Solid understanding of LLMs, AI evaluation fundamentals, and how models process content across different languages.
• Demonstrated experience shipping or evaluating features across multiple languages or locales.
• Experience designing benchmarks, datasets, and human evaluation protocols with a focus on statistical rigor.
• Ability to drive initiatives independently while collaborating effectively within cross-functional, interdisciplinary teams.
• Strong written and verbal communication skills to translate complex findings into actionable guidance.