Machine Learning Engineer, Foundation Model Services

What you'll do

• Build production-grade solutions to serve large foundation models to millions of customers in real time.
• Develop inference systems for cutting-edge model architectures in collaboration with the research team.
• Create tools to identify and resolve performance bottlenecks in inference across different hardware types.
• Optimize large-scale language, vision, and speech models using state-of-the-art technologies.
• Manage high-throughput services at supercomputing scale to ensure low latency for global users.
• Develop and maintain scalable systems using modern programming languages like Golang and Python.
• Deploy and manage machine learning applications on cloud infrastructure using Kubernetes and Docker.

What we're looking for

• Bachelor's degree or higher in Computer Science or a related technical field.
• At least 5 years of industry experience in ML technologies including LLMs, Machine Learning, NLP, Information Retrieval, and Statistics.
• Experience with high throughput services at supercomputing scale.
• Proficiency in running applications on Cloud platforms using Kubernetes and Docker.
• Proficiency in building and maintaining systems written in modern languages like Golang or Python.
• Familiarity with popular ML frameworks such as PyTorch or TensorFlow.
• Familiarity with fundamental Deep Learning architectures including Transformers and Encoder/Decoder models.
• Familiarity with tools like Nvidia TensorRT-LLM, vLLM, DeepSpeed, and Nvidia Triton Server.