Algorithm Evaluation Manager

What you'll do

• Lead, mentor, and grow a team of engineers and data specialists.
• Define and execute evaluation strategies for both Computer Vision and Large Language Models.
• Build robust and scalable evaluation pipelines that reflect real-world model performance.
• Oversee the end-to-end data lifecycle including curation guidelines and quality management.
• Manage large-scale annotation workflows with internal teams and external vendors.
• Perform rigorous data analysis to troubleshoot model regressions and identify algorithmic improvements.
• Communicate progress, bottlenecks, and strategic data needs to leadership and partner teams.

What we're looking for

• A Bachelor of Science degree is required.
• Candidates must have at least 10 years of relevant industry experience.
• At least 2 years of direct people management experience is required.
• Experience in model evaluation, benchmarking, and A/B testing for machine learning models is required.
• Familiarity with the design and architecture of machine learning inference pipelines and infrastructure is required.
• Hands-on experience in data curation strategies and human-in-the-loop annotation processes is required.
• Strong analytical skills to identify trends, biases, and model improvements within large datasets are required.
• Excellent verbal and written communication skills for translating complex technical concepts are required.