Senior Research Scientist, Multimodal Foundation Models and Robotics

What you'll do

• Design and implement novel AI algorithms for general-purpose humanoid robots and embodied agents.
• Develop large-scale AI training and inference methods for multimodal foundation models.
• Optimize and deploy AI models in physical simulations and on actual robot hardware.
• Transfer research findings into production-ready products and services across NVIDIA.
• Train and evaluate models using reinforcement learning, imitation learning, and classical control methods.
• Operate robot hardware, lab equipment, and tools safely to test research milestones.
• Utilize physics simulation frameworks like MuJoCo and Isaac Sim for robot learning.

What we're looking for

• Ph.D. in Computer Science, Engineering, Electrical Engineering, or equivalent research experience.
• 5 years of relevant work or research experience in Multimodal Foundation Models and/or Robotics.
• Hands-on training experience and publications in LLMs, vision-language models, video generative models, or action-based transformers.
• Hands-on training experience and publications in robot learning, including reinforcement learning, imitation learning, or classical control methods.
• Outstanding engineering skills in rapid prototyping and model training frameworks like PyTorch, Jax, or TensorFlow.
• Proficiency in Python is required; C++ and CUDA proficiencies are a big plus (preferred).
• Deep understanding of robot kinematics, dynamics, sensors, and control methods like PID or model predictive control.
• Familiarity with physics simulation frameworks such as MuJoCo and Isaac Sim (preferred).