Big Data Engineer Lead

S&P Global

Confirmed live today High trust

Quick summary

Work type
On-site
Location
New York, NY
Salary
$149,302–$202,267 / yr
Posted
28 days ago
Freshness
Confirmed live today
Closes
Sep 29, 2027

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $171k
This role $176k
$117k $229k
below market most similar roles pay here above market

This role pays less than 53% of similar roles. Most pay $142,350–$199,058 — the blue band above. At the midpoint, this role pays about $176k versus about $171k for comparable roles.

Based on 240 similar postings.

Employer

About S&P Global

S&P Global delivers Essential Intelligence® that shapes decision making. We provide the world’s leading organizations with the right data, connected technologies and expertise they need to move ahead.

S&P Global currently has 39 open roles on FindRole.

Listed pay typically runs $145,245–$200,000 across 36 roles with salary data.

Most-posted roles

View all roles at S&P Global

At a glance

TL;DR · Big Data Engineer Lead

The Big Data Engineer Lead will join the Ratings Organization's Data Services Team to help shape the next generation of the analytics platform. In this role, you will partner with business stakeholders to gather requirements and collaborate with data scientists and software engineers to design, build, and deploy scalable machine learning and generative AI solutions. You will develop platforms, services, and tooling to optimize and fine-tune models while analyzing large datasets to provide data-driven insights. Key technologies include Python, Java, Spring Boot, SQL, and AWS. You will utilize frameworks such as TensorFlow, PyTorch, scikit-learn, and LangChain, while working with Docker, Kubernetes, and Git. The role focuses on solving complex problems involving deep learning, natural language processing, and agentic AI workflows within a large-scale data ecosystem.

What you'll do

  • Partner with business stakeholders to gather requirements and define solution use cases.
  • Design, build, and deploy scalable machine learning and generative AI solutions.
  • Analyze large datasets to develop data-driven insights for business outcomes.
  • Build platforms and tooling to optimize and fine-tune machine learning models.
  • Research and apply emerging AI technologies to enhance existing solutions.
  • Develop technical standards and reusable AI capabilities across the organization.

What we're looking for

  • Experience building AI, machine learning, or data-driven solutions in cloud environments like AWS.
  • Strong Python programming skills using frameworks such as TensorFlow, PyTorch, or scikit-learn.
  • Hands-on experience developing applications with Java and Spring Boot.
  • Experience with containerized and virtualized environments including Docker, Kubernetes, and virtual machines.
  • Experience using version control systems such as Git and modern AI-assisted development tools.
  • Strong knowledge of SQL and experience working with structured and unstructured datasets.
  • Understanding of machine learning concepts including deep learning, natural language processing, and generative AI.
  • Experience building AI agents, autonomous workflows, or agentic AI frameworks.

More like this

Similar roles

Data Engineer, Lead

Booz Allen Hamilton

McLean, VA 10 days ago $77,600–$176,000
Databricks Spark Delta Lake Python SQL Vector Databases ETL ELT Retrieval Augmented Generation Embeddings Data Governance APIs Knowledge Engineering Lakehouse Architecture

Lead Data and Analytics Engineer

AbbVie

North Chicago, IL 7 days ago $109,500–$208,500
Python SQL Apache Spark Apache Kafka Apache Flink Databricks Azure AWS Google Cloud Data Modeling Data Warehousing Data Lakehouse Apache Airflow dbt Prefect Generative AI Retrieval-augmented generation Vector Databases GraphQL SDLC GxP
7+ yrs exp Hybrid

Lead Data Engineer

Capital One Financial

Plano, TX 134 days ago
Python Java SQL Scala TypeScript React Databricks Snowflake Spark Kafka AWS Microsoft Azure Google Cloud Redshift Flink DynamoDB OpenSearch UNIX Linux CI/CD Agile Machine Learning Microservices
4+ yrs exp

Lead Data Engineer

3M

71 days ago
Python SQL Spark Databricks Azure Data Factory Cosmos DB Kafka Airflow MLOps CI/CD Data Lake Event Hub Vector Databases Infrastructure as Code
7+ yrs exp

Lead Data Engineer

Capital One Financial

McLean, VA +3 127 days ago
Python SQL Scala Java AWS Microsoft Azure Google Cloud Redshift Snowflake Spark Kafka Hadoop Hive EMR MapReduce Cassandra MySQL UNIX Linux Agile Machine Learning NoSQL RDBMS
4+ yrs exp

Lead Data Engineer

Capital One Financial

Chicago, IL 111 days ago
Python Spark AWS Databricks Snowflake Airflow Java Scala SQL Redshift Hadoop Kafka NoSQL Cassandra Hive EMR MapReduce UNIX Linux Agile
4+ yrs exp