Senior Databricks Data Engineer

What you'll do

• Design and develop high-throughput, configurable data pipelines using Databricks.
• Build and test end-to-end data ingestion pipelines for streaming, events, and batch processing.
• Develop and deploy scalable ETL pipelines on cloud infrastructure using Databricks SQL and Scala/PySpark.
• Optimize and tune Databricks job performance through query optimization and resource management.
• Perform root cause analysis and corrective maintenance to resolve data and environment issues.
• Manage all aspects of the software development lifecycle including coding, testing, and documentation.
• Troubleshoot DevOps pipelines and Azure services to ensure system reliability.

What we're looking for

• Bachelor's degree in a computer or IT related subject (preferred).
• 7+ years of overall IT experience.
• 5+ years of Data Engineering experience handling large datasets.
• 3+ years of Databricks hands-on experience using SQL and Scala/PySpark.
• 3+ years of experience on cloud-based development using Azure or AWS.
• Experience in the financial industry (preferred).
• Experience with Micro Services, API, and Event Driven Architecture (preferred).
• Experience with agile development methodology (preferred).