Data Engineer

What you'll do

• Design and build scalable data pipelines to provide insights into quantum hardware performance and system reliability.
• Develop and maintain ETL/ELT processes using Python and SQL to ensure high-quality, timely data flow.
• Build and operate orchestration workflows in Apache Airflow for dependency management, retries, and monitoring.
• Integrate diverse technical datasets including hardware telemetry, experiment logs, and cloud operations into trusted analytical assets.
• Consume event-driven data from distributed systems using streaming platforms like Apache Kafka or IBM Event Streams.
• Implement scalable data connectors to contribute to the development of the IBM Quantum Lakehouse.
• Perform complex SQL queries and performance tuning using PostgreSQL and Presto for analytical workloads.
• Manage version control and code reviews using Git while following standard software engineering best practices.

What we're looking for

• Master's Degree (preferred).
• Design, build, and maintain scalable data pipelines and ETL/ELT workflows using Python and pandas.
• Apply advanced SQL skills using PostgreSQL and Presto for complex queries and performance tuning.
• Build and operate orchestration workflows in Apache Airflow including dependency management and monitoring.
• Work with streaming platforms such as Apache Kafka or IBM Event Streams to consume event-driven data.
• Experience with Lakehouse solutions, distributed analytics engines like Spark, and data modeling (preferred).
• Experience with hardware telemetry, infrastructure monitoring, or high-volume operational datasets (preferred).