Data Engineer

What you'll do

• Develop and deploy data pipelines and platforms to organize disparate data sources.
• Design, build, and maintain production-level data pipelines.
• Implement and maintain scalable data platforms for diverse client projects.
• Process data using both streaming methods like Kafka and batch processing methods.
• Perform schema design and query optimization for data lakehouse or warehouse platforms.
• Implement best practices for data quality, testing, and observability.
• Conduct analytical exploration and data examination to support client missions.

What we're looking for

• 5+ years of experience designing, building, and maintaining data pipelines in production environments.
• 3+ years of experience developing with distributed systems such as Spark or PySpark.
• 3+ years of experience with data lakehouse or warehouse platforms, schema design, and query optimization.
• 3+ years of experience processing data using streaming (Kafka or Kinesis) and batch methods.
• Experience in Python and SQL.
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
• Experience in data analytics, containerization (Docker, Kubernetes, EKS), AWS, Databricks, Scala, or Java (preferred).
• Bachelor's degree.