Software Engineer II, AI Performance

What you'll do

• Build automated feedback pipelines to correlate operational outcomes with AI model predictions and agent actions.
• Develop programmatic evaluation frameworks to track model precision, recall, and drift over time.
• Create end-to-end data pipelines linking AI outputs to incident lifecycles to quantify business impact.
• Design and deploy scalable APIs and metrics layers for monitoring AI performance and ROI.
• Build single-pane dashboards to communicate operational health and performance narratives to leadership.
• Develop mechanisms to capture structured user feedback to improve model training cycles.
• Translate complex system telemetry and logs into actionable reliability metrics and business insights.

What we're looking for

• 3+ years of experience in backend and data pipeline engineering, including building scalable APIs and metrics platforms.
• Experience with data platform engineering and distributed processing tools like PySpark, SQL, Pandas, Databricks, or Snowflake.
• Practical understanding of model evaluation concepts such as precision, recall, model drift, and ground-truth labeling systems.
• Hands-on experience with version control (GitHub), containerization (Docker), and cloud-native deployments (AWS/EKS).
• Proficiency with AI-assisted development tools like Cursor or Claude Code.
• Strong analytical skills to translate system telemetry and operational logs into reliability metrics and business insights.
• Excellent cross-functional collaboration skills to partner with SRE and DevOps teams.
• Bachelor’s degree in Computer Science, Engineering, or equivalent experience.
• Experience with observability platforms (Datadog, Grafana) and agentic workflows/foundation models (preferred).
• Knowledge of incident management workflows and SRE metrics (preferred).
• Prior experience in data attribution or performance visibility tooling in high-scale environments (preferred).