Machine Learning Research Engineer, Pretraining Data

What you'll do

• Research, design, and implement novel algorithms for large-scale pretraining data curation, filtering, and deduplication.
• Build and optimize scalable data processing systems capable of handling trillions of tokens across diverse sources.
• Develop machine learning-based approaches for data quality assessment, synthetic data generation, and efficiency optimization.
• Design and execute training experiments to analyze the relationship between data characteristics and model performance.
• Translate research prototypes into efficient, reliable, and production-ready data systems.
• Investigate training dynamics and scaling laws to identify opportunities for improving foundation model capabilities.

What we're looking for

• BS, MS, or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent practical experience.
• Strong programming skills in Python and proficiency in developing production-quality software.
• Experience with large-scale data processing, distributed computing, or machine learning systems.
• Solid understanding of machine learning fundamentals and modern deep learning architectures.
• Strong analytical and problem-solving skills to independently investigate complex technical challenges.
• Experience with LLM pretraining, training data research, or foundation model development (preferred).
• Experience designing and implementing scalable data processing pipelines using Spark, Ray, or similar distributed computing frameworks (preferred).
• Hands-on experience with PyTorch, model training, and experimental evaluation (preferred).