Senior Data Engineer II

Allstate

Confirmed live yesterday High trust
Remote

Quick summary

Work type
Remote
Location
IL
Salary
$90,700–$153,925 / yr
Posted
4 days ago
Freshness
Confirmed live yesterday

Market check

Salary context

Below market

How this pay compares to similar roles

Similar $167k
This role $122k
$77k most similar roles pay here $222k

This role pays less than 91% of similar roles. Most pay $131,991–$202,800 — the shaded band above. At the midpoint, this role pays about $122k versus about $167k for comparable roles.

Based on 240 similar postings.

Employer

About Allstate

The Allstate Corporation is one of the largest publicly held personal lines insurers in the US, widely recognized for its "You're In Good Hands With Allstate®" slogan.

Allstate currently has 35 open roles on FindRole.

Listed pay typically runs $100,000–$170,500 across 33 roles with salary data.

Most-posted roles

View all roles at Allstate

At a glance

TL;DR · Senior Data Engineer II

The Senior Data Engineer Con II joins the Data & Analytics Technology team to lead the design, development, and operation of scalable, high-performing data platforms powering enterprise analytics, reporting, AI, and advanced data products. This role involves building and optimizing batch and streaming data solutions using Apache Spark, Microsoft Fabric, and modern cloud-native technologies. The engineer will implement Medallion Lakehouse architecture patterns to transform complex data into business-ready assets while managing ETL/ELT pipelines and dimensional modeling for reporting and self-service analytics. Key technical requirements include proficiency in Python, SQL, and Delta/Parquet formats, alongside experience with CI/CD, DevSecOps, and Agentic AI-assisted development tools. The role focuses on solving complex data integration challenges, ensuring data quality and governance while managing schema evolution and performance tuning for large-scale analytical workloads across diverse enterprise source systems.

What does a Data Engineer earn?

Median $170950 from 215 postings across 56 companies.

See salary data

What you'll do

  • Design, build, and maintain scalable batch and real-time data pipelines using Apache Spark, SQL, and Python.
  • Architect and implement Medallion Lakehouse data patterns (Bronze, Silver, Gold) for enterprise analytics and AI use cases.
  • Develop and optimize ETL/ELT pipelines to ingest, transform, and curate data from diverse source systems.
  • Design robust dimensional models including fact and dimension tables to support business intelligence solutions.
  • Establish data quality, observability, and governance frameworks through automated validation and monitoring.
  • Optimize Spark workloads for performance, scalability, and cost efficiency across large datasets.
  • Implement CI/CD pipelines and DevSecOps practices to automate testing, deployment, and release management.
  • Lead technical design reviews and provide mentorship to junior and mid-level data engineers.

What we're looking for

  • 7+ years of experience as a Data Engineer building enterprise-scale data platforms and pipelines.
  • Strong hands-on experience with Apache Spark for large-scale distributed data processing.
  • Advanced proficiency in Python, SQL, and modern data engineering development practices.
  • Experience designing and implementing Lakehouse architectures and Medallion Architecture patterns.
  • Strong understanding of dimensional modeling including star schemas, snowflake schemas, fact tables, and dimension tables.
  • Experience with Microsoft Fabric, OneLake, Delta/Parquet formats, or comparable cloud analytics platforms.
  • Experience implementing CI/CD pipelines, source control, Infrastructure as Code (IaC), and automated testing practices.
  • Familiarity with AI-powered engineering tools and Agentic AI-assisted software development practices.

More like this

Similar roles

Senior Data Engineer

Mastercard

Atlanta, GA 4 days ago $115,000$184,000
SQL Python Java Scala Spark Kafka Hadoop AWS Azure GCP CI/CD Infrastructure as Code ETL ELT Data Governance Git DevOps Machine Learning Data Lake Data Warehouse

Senior Data Engineer

CoStar Group

Richmond, VA 32 days ago $133,000$172,000
SQL Python Scala C#/.NET Databricks Spark Snowflake BigQuery Redshift Airflow dbt Kafka AWS Azure GCP Terraform CI/CD ETL ELT Delta Lake Apache Iceberg Apache Hudi Fivetran Debezium Great Expectations S3 Glue EMR Lambda
5+ yrs exp

Senior Data Engineer

Marathon Petroleum

San Antonio, TX +1 117 days ago
Azure Python SQL Spark Databricks Azure Data Factory Data Lake Synapse Analytics Azure Functions ETL Hadoop NoSQL Big Data Data Modeling Data Warehousing DevOps Real-Time Processing
5+ yrs exp

Senior Data Engineer

CoStar Group

San Diego, CA 109 days ago $124,000$190,000
Databricks Apache Spark C# .NET SQL AWS ETL ELT Kafka Kinesis CI/CD DevOps Claude Code Streaming Technologies
5+ yrs exp Hybrid

Senior Data Engineer

Mastercard

O'Fallon, MO 2 days ago $115,000$184,000
Spark Scala Python Java Databricks AWS SQL Hadoop Nifi Oracle PostgreSQL Iceberg Ozone ETL ELT Machine Learning TensorFlow PyTorch Generative AI Data Warehousing

Senior Data Engineer

Microsoft

5 days ago $119,800$234,700
Microsoft Fabric Azure OneLake DAX ETL ELT Data Modeling Medallion Architecture CI/CD Infrastructure-as-Code Purview Data Governance Machine Learning Copilot DevOps Data Engineering
4+ yrs exp Hybrid