Senior Data Engineer II

Allstate

Confirmed live yesterday High trust
Remote

Quick summary

Work type
Remote
Location
IL
Salary
$90,700–$153,925 / yr
Posted
50 days ago
Freshness
Confirmed live yesterday

Market check

Salary context

Below market

How this pay compares to similar roles

Similar $175k
This role $122k
$76k most similar roles pay here $226k

This role pays less than 91% of similar roles. Most pay $139,654–$211,200 — the shaded band above. At the midpoint, this role pays about $122k versus about $175k for comparable roles.

Based on 240 similar postings.

Employer

About Allstate

The Allstate Corporation is one of the largest publicly held personal lines insurers in the US, widely recognized for its "You're In Good Hands With Allstate®" slogan.

Allstate currently has 36 open roles on FindRole.

Listed pay typically runs $100,000–$170,500 across 36 roles with salary data.

Most-posted roles

View all roles at Allstate

At a glance

TL;DR · Senior Data Engineer II

Senior Data Engineer Con II joins the Data & Analytics Technology team to design, build, and operate scalable, reliable, and high-performing data pipelines supporting enterprise analytics, reporting, and advanced data use cases. The role involves developing batch and streaming solutions using Apache Spark and modern cloud platforms like Microsoft Fabric and OneLake. Day-to-day responsibilities include creating ETL/ELT workflows to transform complex raw data into trusted datasets, managing data schemas, and implementing CI/CD pipelines for automated testing and deployment. The candidate will utilize Python, SQL, and distributed processing concepts to ensure data quality and performance across large datasets. This position focuses on solving the technical challenge of transforming diverse source data into analytics-ready formats while collaborating with data scientists and product teams to enable data-driven decision making within a complex enterprise environment.

What does a Data Engineer earn?

Median $183138 from 214 postings across 58 companies.

See salary data

What you'll do

  • Design, build, and maintain scalable batch and streaming data pipelines using Apache Spark and cloud technologies.
  • Develop and optimize ETL/ELT workflows to transform raw data from diverse sources into analytics-ready datasets.
  • Implement data modeling and transformation logic to support reporting, dashboards, and machine learning workloads.
  • Manage data processing workloads within modern lakehouse platforms like Microsoft Fabric and OneLake.
  • Ensure data quality and consistency by implementing validation checks, monitoring, and reconciliation processes.
  • Optimize Spark jobs for performance, cost efficiency, and scalability across large, complex datasets.
  • Develop reusable frameworks, libraries, and standardized patterns to improve engineering productivity.
  • Implement CI/CD pipelines for automated testing, deployment, and rollback of data workloads.

What we're looking for

  • Strong experience as a Data Engineer building and operating production data pipelines.
  • Hands-on experience with Apache Spark for large-scale data processing.
  • Proficiency in Python, SQL, and data transformation best practices.
  • Experience with cloud-based data platforms and storage like Data Lakes or Lakehouse architectures.
  • Familiarity with Microsoft Fabric, OneLake, or similar analytics platforms is a strong plus.
  • Experience designing and optimizing data models for analytical workloads.
  • Understanding of distributed data processing concepts, performance tuning, and fault tolerance.
  • Experience with CI/CD, version control, and infrastructure-as-code concepts.

More like this

Similar roles

Senior Data Engineer

Plaid

New York, NY 8 days ago $190,800$238,800
SQL Python Golang DBT Airflow Redshift Snowflake Databricks Spark Kafka Retool Atlan Mode Data Lake Data Warehouse realtime pipelines Schema Design
4+ yrs exp

Senior Data Engineer

Plaid

New York, NY +2 8 days ago $190,800$238,800
SQL Python Golang DBT Airflow Redshift Snowflake Databricks Spark Kafka Retool Atlan Mode Data Lake Data Warehouse realtime pipelines Schema Design
4+ yrs exp

Senior Data Engineer

General Motors (GM)

Austin, TX 10 days ago $119,200$175,450
Azure Databricks Apache Spark Delta Lake Python SQL Scala Azure AWS GCP Kubernetes CI/CD Infrastructure as Code NoSQL JSON Parquet Avro XML Unix Linux Data Modeling ETL ELT
5+ yrs exp Hybrid

Solutions Engineer, Data Engineering

SHI International

Remote (TX) 15 days ago $156,000$181,000
Microsoft Azure Microsoft Fabric Azure Data Factory Azure Synapse Analytics Databricks SQL Server Azure SQL ADLS Gen2 ETL ELT Data Modeling CI CD Source Control Power BI Data Lakehouse Data Governance Infrastructure as Code
Remote

Senior Data Engineer

General Motors (GM)

Austin, TX 77 days ago $133,000$188,600
Azure Databricks Python SQL Spark Scala MLflow Unity Catalog CI/CD Git Kafka Event Hub Java ETL ELT Structured Streaming IoT
5+ yrs exp Hybrid

Senior Data Engineer

Adobe

San Francisco, CA 45 days ago $163,200$236,400
Databricks SQL Python PySpark Airflow Unity Catalog Azure RAG LLMs ETL Feature Stores Embeddings
5+ yrs exp