Senior AI Platform Engineer, Data and Systems

What you'll do

• Design and build streaming-first data pipelines to reduce end-to-end latency from hours to minutes.
• Own and extend the ML Attribute Store to provide low-latency online feature serving and batch aggregation.
• Build MCP-compatible Agent Data APIs and tool servers for autonomous AI systems to query the lakehouse.
• Develop an agentic framework for automated anomaly detection, pipeline self-healing, and root cause analysis.
• Drive operational excellence through observability, incident response automation, performance tuning, and cost optimization.
• Translate platform capabilities into self-serve infrastructure to reduce engineering toil for non-platform teams.
• Use AI-powered developer tools to accelerate personal and team engineering velocity.

What we're looking for

• 6+ years of experience in data platform engineering, distributed systems, or backend infrastructure at scale.
• Deep hands-on experience with Apache Spark, Databricks, Delta Lake, or equivalent lakehouse technologies.
• Proven track record building and operating large-scale pipelines processing billions of events daily with sub-hour latency SLAs.
• Strong experience with streaming systems such as Kafka, Kinesis, Flink, Spark Structured Streaming, or Delta Live Tables.
• Proficiency in Python and/or Scala with required SQL fluency; Java or Go is a plus (preferred).
• Experience with cloud platforms (AWS or Azure), containerization (Docker, Kubernetes), and CI/CD for data pipelines.
• Production experience integrating LLMs into engineering workflows and hands-on experience with agentic AI frameworks like LangChain or CrewAI.
• BS/MS in Computer Science, Engineering, or equivalent practical experience (preferred).