Senior Software Engineer, Apple Data Platform

What you'll do

• Design and implement native operators and expressions using Apache DataFusion and Apache Arrow.
• Extend Comet's integration across various Spark versions to meet Apple-scale requirements.
• Accelerate Apache Iceberg and Parquet read and write operations.
• Optimize runtime efficiency, memory usage, and I/O for data center and cloud environments.
• Contribute code, perform reviews, and participate in architectural discussions for Apache Spark and DataFusion.
• Build clean interfaces and abstractions to integrate native query execution into broader data workflows.
• Profile and benchmark production workloads to ensure system reliability and performance.

What we're looking for

• 3+ years of professional software engineering experience.
• A proven track record of building distributed systems.
• Contributor or committer to Apache Spark and/or Apache DataFusion, Comet.
• BS, MS, or PhD in Computer Science or a related field, or equivalent practical experience.
• Experience with columnar or vectorized query execution, or with performance profiling and optimization of query engines (preferred).
• Good knowledge of Apache Iceberg and Apache Parquet internals (preferred).
• Experience working on or building Spark connectors to external data sources (preferred).
• Knowledge of Kubernetes, or other compute platforms (preferred).