The Commerce Data Pipelines team within Disney’s Data organization seeks a Data Engineer II to establish and maintain foundational data pipelines in Scala and Python/Spark. This role involves expanding existing platforms, implementing Lakehouse architecture, and developing shared libraries to ensure consistent functionality across all data pipelines. Key responsibilities include extending metadata parsing capabilities, building new integrations with internal and external APIs, and collaborating closely with product managers and architects to drive platform success. The tech stack includes Airflow, Spark, Databricks, Delta Lake, Snowflake, and Scala/Python, requiring strong algorithmic problem-solving skills and hands-on experience with distributed processing systems like Spark and data pipeline orchestration tools such as Airflow. Ideal candidates will have a background in developing large-scale data pipelines and a deep understanding of cloud providers like AWS.
Maintain and expand Commerce Data platform data pipelines using Scala and Python/Spark.
Extend functionality of the Commerce Data platform by building new integrations with APIs.
Implement Lakehouse architecture to shift towards a centralized data platform approach.
Develop shared libraries in Scala and Python for consistent business logic across pipelines.
Ensure high operational efficiency and quality of datasets to meet SLAs and stakeholder needs.
What we're looking for
3+ years of data engineering experience developing large data pipelines
Proficient in Scala and Python for complex business logic abstraction
Hands-on experience with distributed processing systems like Spark and Airflow
Strong SQL skills for analyzing complex datasets
Experience with cloud providers (AWS, Databricks) and infrastructure as code
Familiarity with Lakehouse architecture and Delta Lake technologies
Active participation in agile/scrum ceremonies and documentation practices
Market check
Salary context
This
$112,000–$150,100
range sits above
41%
of similar postings on FindRole.
Peer median band
$99,000–$189,600
Median floor and ceiling across peers.
Typical midpoint (25–75%)
$126,800–$193,415
Middle half of comparable postings.
Based on 239 comparable postings.*
* 240 is the maximum number of comparable postings sampled.
Employer
About The Walt Disney Company
The Walt Disney Company is a diversified global entertainment and media enterprise operating in segments including Disney Parks, Experiences and Products; Entertainment (ABC, Hulu, Disney+); and ESPN. Industry: Entertainment & Media
The Walt Disney Company currently has
118 open roles
on FindRole.
Listed pay typically runs
$141,900–$190,300
across 110 roles with salary data.