Software Engineer L5/L6, Model Evaluations & Data Curation

What you'll do

• Design and build shared data curation infrastructure to replace ad-hoc notebooks with reusable components and workflows.
• Build scalable LLM-driven pipelines to transform raw sources into training and evaluation data using large-scale batch inference.
• Develop sampling strategies to ensure coverage, diversity, and balance across content and member segments.
• Create filtering and quality-control methods using LLM-as-judge scoring, deduplication, and validation techniques.
• Partner with researchers to measure how specific data curation choices impact downstream model performance.
• Manage curated datasets as versioned artifacts with explicit lineage and reproducibility for team discovery.
• Drive the adoption of shared curation practices across the MEDC and partner modeling teams.

What we're looking for

• Strong software engineering skills in Python to build reusable infrastructure, libraries, or frameworks.
• Experience building LLM-driven data generation or transformation pipelines, including synthetic data and batch inference at scale.
• Hands-on experience with data quality methods such as sampling strategies, filtering, deduplication, and LLM-as-judge scoring.
• Modeling intuition to understand how data choices affect model behavior and the ability to design experiments to measure it.
• Experience with distributed data processing using Spark, Ray, or similar technologies.
• Excellent collaboration skills with researchers, data scientists, and platform teams.
• Experience with LLM evaluation systems (preferred).
• Technical leadership experience setting direction for multi-engineer efforts (preferred).