Sr. Databricks Data Engineer, Vice President, Hybrid

State Street

Hybrid Actively hiring
Quincy, MA · Princeton, NJ Posted 18 days ago $120,000–$202,500 / year
View original post Log in to save

At a glance

AI generated

TL;DR

The Senior Databricks Data Engineer will join a dynamic and agile scrum team to design, build, and maintain complex data pipelines on cloud-based infrastructure using Azure and Cosmos DB. This role involves developing ETL processes for data ingestion, transformation, and loading into data lakes, as well as creating custom high-throughput frameworks/libraries. The candidate must have extensive experience with Databricks, Delta Lake, Apache Spark, and be proficient in Scala and SQL within the Databricks environment. Additionally, they will monitor and troubleshoot pipelines, implement data quality checks, and work effectively in a multi-developer setting using Git for version control. Ideal candidates possess 10+ years of big data pipeline experience, including at least five years with Databricks and Azure services, and have a background in the financial industry.

Skills

Azure Databricks Delta Lake Apache Spark Scala Kubernetes Docker Git Shell Scripting UNIX Snowflake CI/CD SQL

What you'll do

  • Design and build end-to-end data pipelines using Azure and Cosmos DB.
  • Develop ETL processes for ingesting, transforming, and loading data into data lakes.
  • Work on Databricks and data warehousing concepts to enhance platform capabilities.
  • Monitor and troubleshoot data pipelines to resolve issues efficiently.
  • Implement data quality checks and validations to ensure data accuracy.

What we're looking for

  • 10+ years of overall Bigdata data pipeline experience.
  • 5+ years of hands-on Databricks experience and strong understanding of the platform.
  • Extensive cloud-based development experience with Azure Services, DevOps, Kubernetes, and Docker.
  • Proficient in Data Warehousing platforms like Databricks, Delta Lake, and Apache Spark.
  • Experience designing and implementing ETL processes for data ingestion, transformation, and loading into data lakes.
  • Strong critical thinking, communication, and problem-solving skills; ability to work collaboratively within an agile team.

Market check

Salary context

This $120,000–$202,500 range sits above 48% of similar postings on FindRole.

Peer median band

$120,000–$206,390

Median floor and ceiling across peers.

Typical midpoint (25–75%)

$133,900–$209,000

Middle half of comparable postings.

Based on 240 comparable postings.

* 240 is the maximum number of comparable postings sampled.

Employer

About State Street

State Street Corporation is one of the world''s largest custodian banks and asset managers, providing investment servicing, investment management, and investment research to institutional investors. Industry: Financial Services & Asset Custody

State Street currently has 133 open roles on FindRole.

Listed pay typically runs $110,000–$180,000 across 131 roles with salary data.

Most-posted roles

View all roles at State Street

More like this

Similar roles

Databricks Tech Lead - Vice President

Citi

Remote (3800 Citigroup Center Drive Building F Tampa, US) 11 days ago $113,840–$170,760
AWS Databricks Python SQL Terraform CloudFormation CI/CD S3 Glue Athena SQS Lambda Delta Lake Spark SQL
Remote

Sr. Databricks Data Engineer, Onsite, AVP

State Street

US 22 days ago $90,000–$157,500
Databricks Scala PySpark Azure AWS Microservices APIs Event-Driven Architecture CI/CD Agile_methodology SQL Data_Lakehouse

Data Engineer - Assistant Vice President

Citi

Remote (3800 Citigroup Center Drive Building C Tampa, US) 23 days ago $96,960–$145,440
Python Java Scala Hadoop Snowflake Databricks SQL Kubernetes Spark Kafka Airflow Terraform AWS Google Cloud
Remote

Sr. Data Engineer - Assistant Vice President

Citi

Remote (6460 Las Colinas Blvd Irving, US) 10 days ago
Hadoop Spark Kafka Hive Parquet Avro Python Scala Java Databricks Microservices AI ML Deep Learning NLP SQL Docker Kubernetes Data Mesh Starburst
Remote

SR. Data Engineer - Assistant Vice President

Citi

Remote (6460 Las Colinas Blvd Irving, US) 10 days ago
Hadoop Spark Kafka Hive Python Scala Java Databricks ETL ELT Microservices AI ML DeepLearning NLP SQL Docker Kubernetes AWS Azure GCP DataMesh Starburst
Remote