Senior Data Engineer

What you'll do

• Design, develop, and optimize scalable data pipelines and cloud-based solutions using Azure services like Synapse and Databricks.
• Architect complex systems to process and store high-volume datasets while ensuring data security and privacy.
• Perform complex data extraction, ingestion, and transformation processes using Python, SQL, and Spark.
• Provide technical expertise during critical incidents and resolve complex infrastructure problems.
• Implement best practices for data quality, governance, and consistent processing methods across all projects.
• Oversee quality assurance and testing for all deployed data solutions.
• Mentor less experienced engineers and contribute to engineering standards and reusable patterns.

What we're looking for

• Bachelor's Degree in Information Technology or a related field.
• 5+ years of relevant data engineering experience.
• Proficiency in Python, SQL, Spark, and orchestration tools like Azure Data Factory or Databricks Workflows.
• Experience with Azure services including Data Lake, Synapse Analytics, Azure Functions, and Databricks.
• Familiarity with big data technologies such as Hadoop and distributed computing for large volume processing.
• Knowledge of cloud platforms (AWS, Azure, or GCP) and associated storage and analytics services.
• Expertise in data modeling, integration, lifecycle management, and real-time processing.
• Ability to implement data governance, security, privacy measures, and quality management standards.