Senior Data Engineer

What you'll do

• Build and maintain high-performance streaming and batch data pipelines to power AI applications with low-latency ingestion.
• Implement embedding generation workflows, vector store integrations, and retrieval pipelines for RAG systems and AI assistants.
• Develop scalable storage and retrieval patterns focused on cost-efficient architecture and production performance.
• Integrate data pipelines with shared AI platform services including agent frameworks, registries, and feature stores.
• Embed end-to-end observability into data systems using metrics, structured logging, automated alerts, and drift detection.
• Implement robust data quality validation, schema evolution safeguards, and governance compliance controls.
• Own the full development lifecycle including prototyping, implementation, testing, deployment, and documentation.
• Influence system design through technical proposals and high-quality code to drive production-ready AI capabilities.

What we're looking for

• Bachelor’s degree in computer science or a related quantitative field, or equivalent practical experience.
• At least 5 years of data engineering experience.
• At least 1 year of experience in a lead or senior technical role.
• Expert level skill in either Python or Java, with strong proficiency in the other.
• Experience building and scaling streaming data pipelines in large-scale distributed environments using tools like Kafka, Flink, Spark, or Kinesis.
• Experience with embedding pipelines and vector stores such as Pinecone, Weaviate, FAISS, or pgvector.
• Proficiency in SQL and experience with workflow orchestration tools like Airflow or Dagster.
• Strong knowledge of data modeling, storage optimization, and retrieval patterns for large-scale systems.