Data Scientist, AI/ML Model Quality

What you'll do

• Curate and maintain gold-standard ground-truth datasets for ML and GenAI model evaluation.
• Audit training data for systemic bias and fairness gaps before model deployment.
• Define and report key data quality metrics such as completeness, accuracy, and timeliness.
• Design automated data quality rules and thresholds integrated into CI/CD workflows.
• Define and track ML observability metrics to identify performance issues and feature drift.
• Develop dashboards and reporting workflows to provide real-time views of model health.
• Analyze telemetry across GenAI workflows to monitor output coherence, latency, and regression patterns.
• Identify domain-specific failure modes in GenAI systems and provide actionable recommendations for engineering teams.

What we're looking for

• A Bachelor's degree is required, while a Master's or PhD in a quantitative field is preferred.
• 3+ years of experience in data science or a related role focused on data quality, model evaluation, or ML observability.
• Proficiency in Python (Pandas, NumPy, Scikit-learn) and SQL for complex data analysis and metric creation.
• Experience querying and analyzing large-scale datasets using distributed computing frameworks like PySpark or Spark.
• Solid understanding of statistical methods including hypothesis testing, distribution analysis, and data drift detection.
• Experience defining and tracking ML model health metrics in production environments.
• Familiarity with GenAI or LLM systems, including common quality failure modes and telemetry instrumentation.
• Experience with data visualization tools, LLM evaluation frameworks, or working with privacy-constrained data (preferred).