Senior Data Engineer, Services Special Projects

What you'll do

• Build and implement batch and streaming ETL/ELT pipelines to process multimodal data from diverse sources.
• Develop and maintain Kafka-based ingestion and processing pipelines for reliable data delivery into the data lake.
• Create robust logical and physical data models optimized for dimensional modeling, versioning, and storage.
• Define and enforce data quality checks, SLAs, and observability standards to ensure data accuracy and trust.
• Integrate raw signals with metadata and attribution to power downstream analytics, billing, and optimization use cases.
• Implement methodologies for data lineage, metadata management, schema governance, and security standards.
• Deliver solutions featuring logging, anomaly detection, data validation, and automated cleaning for continuous improvement.
• Deploy and optimize LLMs or ML models directly in production inference runtimes for real-time data enrichment.

What we're looking for

• 10+ years of experience in data engineering, including building and maintaining large-scale ETL/ELT data pipelines.
• Proficiency in data modeling, especially dimensional modeling, and designing schemas optimized for analytics and reporting.
• Experience with SQL/NoSQL databases including Postgres, Cassandra, and Redis.
• Strong experience with distributed data processing frameworks including Apache Spark and parallel processing frameworks like BigTable/Hadoop.
• Strong software engineering fundamentals and proven experience with Scala and Java.
• Hands-on experience with Apache Kafka, Iceberg, Flink, and workflow orchestration tools like Apache Airflow and Beam.
• Experience with AWS services, containerization (Docker, Kubernetes/EKS), CI/CD tooling, and Python/PySpark.
• Masters Degree.