Lead Data Engineer

What you'll do

• Lead the architecture and development of scalable, secure data pipelines for AI/ML workloads.
• Own end-to-end data engineering processes including ingestion, transformation, storage, quality, and monitoring.
• Partner with data scientists and ML engineers on model features, training pipelines, and deployment.
• Drive best practices in data modeling, orchestration, versioning, and performance optimization.
• Mentor junior engineers and contribute to technical roadmaps and solution patterns.
• Ensure data governance, lineage, and compliance standards are met across all platforms.
• Support real-time and batch processing frameworks in production environments.

What we're looking for

• Bachelor's degree or higher in computer science from an accredited institution.
• Seven or more years of data engineering experience, including leading technical initiatives.
• Strong expertise in Python, SQL, and distributed data systems like Spark, Databricks, or Synapse (preferred).
• Experience building AI/ML ready data pipelines, including feature stores and model training data flows (preferred).
• Hands-on experience with cloud platforms, specifically Azure Data Lake, Data Factory, Databricks, and Cosmos DB (preferred).
• Strong understanding of ML concepts, lifecycle, and MLOps practices (preferred).
• Proven experience with workflow orchestration tools like Airflow, Data Factory, or Synapse Pipelines (preferred).
• Experience with streaming platforms, vector databases, or LLM-oriented data pipelines (preferred).