Data Scientist, AI/ML Model Quality

What you'll do

• Curate and maintain gold-standard datasets for model evaluation across ML and GenAI systems.
• Audit training data for bias and fairness gaps before deployment.
• Define and track key data quality metrics for engineering and leadership reporting.
• Design automated data quality rules and thresholds in collaboration with Data Engineering.
• Define observability metrics to monitor silent degradation, feature drift, and model performance.
• Develop observability dashboards for real-time monitoring of model health across systems.
• Identify degradation patterns in GenAI systems through systematic telemetry analysis.

What we're looking for

• 3+ years of experience in data science focusing on data quality and model evaluation.
• Strong proficiency in Python for complex data analysis and validation using Pandas, NumPy, Scikit-learn.
• Experience with large-scale dataset querying and analysis using distributed computing frameworks like PySpark or Spark.
• Solid understanding of statistical methods including hypothesis testing, distribution analysis, and data drift detection.
• Familiarity with defining and tracking ML model health metrics in production environments.
• Knowledge of GenAI systems and common quality failure modes.