Senior Machine Learning Engineer, Applied Science Data Frameworks

What you'll do

• Build and maintain distributed training data loaders for multi-source data ingestion and real-time transformations.
• Implement feature enrichment pipelines and dataset registry systems for multimodal model training.
• Develop batch inference pipelines for large-scale feature extraction using distributed GPU clusters.
• Build data processing systems using Apache Ray, Spark, or DuckDB for SQL-based ingestion and Arrow storage.
• Support semantic search capabilities and vector database infrastructure for dataset discovery and retrieval.
• Manage CI/CD infrastructure including self-hosted runners, Docker builds, and automated testing pipelines.
• Optimize data pipeline performance regarding startup latency, throughput, memory footprint, and GPU utilization.
• Write reusable framework components, SDKs, and documentation to accelerate platform adoption across modeling teams.

What we're looking for

• 5-6 years of professional experience building and operating distributed systems or data infrastructure in production environments.
• Proficiency in Python and strong software engineering fundamentals including system design, data structures, and algorithms.
• Solid understanding of distributed computing concepts and experience with frameworks like Apache Spark, Ray, Dask, or equivalent.
• Familiarity with cloud platforms (AWS or Azure) and data platforms such as Databricks or Spark.
• Basic familiarity with MLOps practices including CI/CD pipelines, containerization (Docker), and deployment automation.
• Familiarity with ML frameworks such as PyTorch or TensorFlow; hands-on ML experience is a plus (preferred).
• Familiarity with batch inference architectures and large-scale data processing patterns is a plus (preferred).
• Bachelor's degree in Computer Science, Engineering, or a related field; MS is a plus (preferred).