Machine Learning Engineer, Foundation Model Services

What you'll do

• Build production-grade solutions to serve foundation models to millions of customers in real time.
• Prototype and develop inference for cutting-edge model architectures with the Foundation Model Research team.
• Build tools to identify and analyze performance bottlenecks in inference across different hardware and use cases.
• Optimize billions of parameters for language, vision, and speech models using state-of-the-art technologies.
• Develop and maintain high-throughput systems using modern languages like Golang and Python.
• Deploy and manage machine learning applications on cloud platforms using Kubernetes and Docker.
• Implement large-scale inference solutions using frameworks like PyTorch, TensorFlow, or TensorRT-LLM.

What we're looking for

• 5 year+ industry experience in ML technologies including LLMs, Machine Learning, NLP, Information Retrieval, and Statistics.
• Experience with high throughput services particularly at supercomputing scale.
• Proficiency in running applications on Cloud (AWS / Azure or equivalent) using Kubernetes and Docker.
• Proficiency in building and maintaining systems written in modern languages such as Golang or Python.
• Bachelor’s degree or higher in Computer Science or a related technical field.
• Familiarity with popular ML Frameworks like PyTorch or TensorFlow (preferred).
• Familiarity with fundamental Deep Learning architectures such as Transformers and Encoder/Decoder models (preferred).
• Familiarity with Nvidia TensorRT-LLM, vLLM, DeepSpeed, or Nvidia Triton Server (preferred).