Software Engineer, Evaluation

What you'll do

• Design and implement evaluation benchmarks, metrics, and test suites to measure model capabilities.
• Develop evaluation methods that capture model behavior in real product settings and predict user-perceived quality.
• Research and apply state-of-the-art evaluation techniques including scoring frameworks and model-based judging.
• Build reusable tools, scorer libraries, and analysis frameworks that scale across the benchmark portfolio.
• Design and execute rigorous experiments to compare model capabilities and perform detailed gap analysis.
• Tune evaluation infrastructure to improve operational controls, auditability, debug trails, and workflow scaling.
• Provide evaluation insights to model training and data teams to inform training strategies and product improvements.

What we're looking for

• 3+ years of professional software engineering experience building and operating production systems.
• BS, MS in Computer Science, Computer Engineering, or an equivalent degree with applicable experience.
• Experience designing and scaling distributed systems including load balancing, active-active topologies, and capacity planning.
• Experience with frontend web development using React and modern JavaScript frameworks.
• Experience with RESTful API design, SQL, and NoSQL databases.
• Fluent use of AI-assisted development tools like coding agents and code review assistants.
• Experience working on large, cross-functional, and multi-team projects (preferred).
• Proven ability to operate independently on ambiguous, open-ended problems where the right answer is not obvious (preferred).