Software Engineer II, Big Data

What you'll do

• Design and implement robust data infrastructure using Spark and Scala within AWS environments.
• Evolve core data pipelines to ensure they scale efficiently for high-growth demands.
• Select and implement optimal storage engines and formats based on performance and cost factors.
• Build and implement knowledge graphs accessible through both batch processing and APIs.
• Optimize AWS resources while designing systems for large-scale data handling.
• Implement automated data quality checks to ensure accuracy in production environments.
• Develop APIs backed by complex, relationship-heavy datasets for product use.

What we're looking for

• Bachelor's degree in Computer Science or a related field.
• Production data engineering experience.
• Proficiency in Spark and Scala to build data infrastructure.
• Strong proficiency in AWS services and SQL for data manipulation.
• Experience delivering APIs backed by relationship-heavy datasets.
• Familiarity with data lakes, cloud warehouses, and storage formats.
• Demonstrated ability to use AI tools while maintaining high standards of verification and integrity.
• Excellent written and verbal communication skills.