Staff Security Data Engineer

What you'll do

• Own end-to-end designs for the Security Data Platform architecture using Databricks, AWS, Spark, and Delta Lake.
• Build and evolve distributed batch and streaming pipelines for various security telemetry and logs.
• Write maintainable Python and SQL code to develop complex platform features and refactor unreliable code.
• Build and maintain reusable Terraform modules for AWS and Databricks infrastructure components.
• Design layered lakehouse data models and establish data contracts, quality checks, and governance policies.
• Engineer and operate Airflow and MWAA orchestration for pipeline dependencies, retries, and automated recovery.
• Lead diagnosis and resolution of complex production issues across Spark, storage, and cloud infrastructure.
• Profile workloads to drive measurable cost and performance improvements through query tuning and compute right-sizing.

What we're looking for

• Must have 10 or more years of experience building and operating production data platforms in data or software engineering.
• Must have a record of personally delivering complex distributed systems and leading full-system design and implementation across team boundaries.
• Must possess deep production expertise in Spark, Databricks, Delta Lake, and lakehouse architecture.
• Must have strong Python and advanced SQL skills for data modeling, profiling, and query optimization.
• Must have strong AWS experience across storage, compute, networking, and IAM.
• Must have production Terraform experience designing, testing, and maintaining reusable infrastructure modules.
• Must have experience with Airflow, MWAA, or equivalent orchestration, and with production observability, data quality, and CI/CD.
• Experience with security telemetry, OCSF schemas, and Unity Catalog governance (preferred).