Engineering Manager, Machine Learning Platform Technologies

What you'll do

• Lead a team of engineers to operate a petabyte-scale distributed web crawl platform.
• Define the architectural vision and technical roadmap for multi-service data collection pipelines.
• Manage the end-to-end delivery of large, multi-quarter projects including scoping and staffing.
• Translate real-world knowledge requirements from product and ML teams into engineering specifications.
• Oversee the reliability, scalability, and feature velocity of mission-critical infrastructure.
• Conduct system design reviews and maintain high engineering quality standards.
• Lead production incident war rooms to resolve critical system issues.
• Align infrastructure investments with strategic AI and search initiatives across the company.

What we're looking for

• 8+ years of software engineering experience.
• 3+ years of engineering management experience leading distributed systems engineers in a production environment.
• Experience designing, building, and operating large-scale distributed systems.
• Experience with cloud infrastructure at scale, AWS or equivalent, Kubernetes, and IaC tooling.
• Familiarity with data pipeline infrastructure, streaming (Flink), batch (Spark), and columnar storage (Iceberg or equivalent).
• Strong programming background in one of the programming languages, preferred in Rust, Scala, or Go (preferred).
• BS or MS in Computer Science or equivalent experience.
• Prior experience with web crawl systems (URL frontier management, politeness, rate limiting, crawl scheduling) (preferred).