Staff Data Engineer

What you'll do

• Design, build, and optimize large-scale data pipelines for both batch and real-time processing.
• Develop and maintain distributed data systems that are highly available, scalable, and resilient.
• Architect and implement secure data platforms using big data technologies like Hadoop and Spark.
• Build and optimize ETL/ELT pipelines to ensure high data quality, reliability, and performance.
• Implement automation, monitoring, and observability for data pipelines across the entire lifecycle.
• Lead proof-of-concept initiatives to evaluate and adopt new data engineering technologies.
• Mentor junior engineers and foster a culture of continuous learning and accountability.

What we're looking for

• Must have a Bachelor's degree with experience, an Advanced degree with experience, or a PhD.
• Experience in big data engineering including Hadoop, Apache Spark, Python, and SQL.
• Proficiency in creating and managing large-scale data pipelines and machine learning models.
• Experience developing ETL processes, maintaining Spark pipelines, and productizing AI/ML models.
• Proficiency in technologies such as Kafka, Redis, Flink, TensorFlow, Triton, and AWS services.
• Skilled in Unix/Shell or Python scripting and scheduling tools like Airflow and Control-M.
• Familiarity with Agile development, TDD, CI/CD, and various databases.
• Experience in Docker and Kubernetes for containerization (preferred).