Senior Machine Learning Engineer, Applied Science Data Frameworks

What you'll do

• Build and maintain distributed training data loaders for multi-source ingestion and real-time transformations at petabyte scale.
• Develop feature enrichment pipelines and dataset registry systems to support multimodal model training across various media types.
• Construct batch inference pipelines for large-scale feature extraction using distributed GPU clusters with fault tolerance.
• Develop data processing systems using frameworks like Apache Ray, Spark, or DuckDB for SQL-based ingestion and storage.
• Support semantic search capabilities and vector database infrastructure for dataset discovery and embedding-based retrieval.
• Manage CI/CD infrastructure including Docker image builds, automated testing pipelines, and deployment automation for ML systems.
• Optimize data pipeline performance regarding startup latency, throughput, memory footprint, and GPU utilization.
• Create reusable framework components, SDKs, and documentation to accelerate platform adoption across modeling teams.

What we're looking for

• 5-6 years of professional experience building and operating distributed systems or data infrastructure in production environments.
• Proficiency in Python and strong software engineering fundamentals including system design, data structures, and algorithms.
• Solid understanding of distributed computing concepts with experience using frameworks like Apache Spark, Ray, Dask, or equivalent.
• Familiarity with cloud platforms (AWS or Azure) and data platforms such as Databricks or Spark.
• Basic familiarity with MLOps practices including CI/CD pipelines, containerization (Docker), and deployment automation.
• Familiarity with ML frameworks such as PyTorch or TensorFlow; hands-on ML experience is a plus.
• Familiarity with batch inference architectures and large-scale data processing patterns is a plus.
• Bachelor's degree in Computer Science, Engineering, or a related field; MS is a plus.