Data Scientist, AI/ML Model Quality

What you'll do

• Curate and maintain gold-standard ground-truth datasets for ML and GenAI model evaluation.
• Audit training data for systemic bias and fairness gaps before deployment.
• Define and report key data quality metrics such as completeness, accuracy, and timeliness.
• Design automated data quality rules and thresholds integrated into CI/CD workflows.
• Define and track ML observability metrics to detect performance issues and feature drift.
• Develop dashboards and reporting workflows to provide real-time views of model health.
• Analyze telemetry across GenAI workflows to monitor output coherence, latency, and regression patterns.
• Identify failure modes in GenAI systems and translate findings into actionable recommendations.

What we're looking for

• A Bachelor's degree is required, while a Master's or PhD in a quantitative field is preferred.
• 3+ years of experience in data science with a focus on data quality, model evaluation, or ML observability.
• Proficiency in Python (Pandas, NumPy, Scikit-learn) and SQL for complex data analysis and metric creation.
• Experience querying and analyzing large-scale datasets using distributed computing frameworks like PySpark or Spark.
• Solid understanding of statistical methods including hypothesis testing, distribution analysis, and data drift detection.
• Experience defining and tracking ML model health metrics and observability in production environments.
• Familiarity with GenAI or LLM systems, including common quality failure modes and telemetry instrumentation.
• Strong communication skills to translate complex technical findings into actionable insights for non-technical stakeholders.
• Experience with data visualization tools, LLM evaluation frameworks, or synthetic data generation (preferred).
• Experience with ML monitoring platforms, privacy-constrained data, or financial services (preferred).