Senior System Software Engineer, Dynamo-Triton Inference Server

What you'll do

• Develop world-class GPU-accelerated AI inference serving software.
• Drive the convergence of Triton Inference Server and NVIDIA Dynamo stacks into a unified platform.
• Ensure feature parity for both Large Language Model and non-LLM workloads.
• Build robust software designed for production server and cloud environments.
• Optimize and balance prediction throughput and latency for high-performance inference.
• Develop and adopt next-generation inference technologies.
• Contribute to the open source deep learning software engineering community.

What we're looking for

• MS or PhD in Computer Science or a relevant field, or equivalent experience.
• 12+ years of professional experience working on deep learning software.
• Excellent Rust and C++ skills with familiarity with Python.
• Strong programming and software design skills including debugging, performance analysis, and test design.
• Experience with high-scale distributed systems and ML systems.
• Strong communication skills and the ability to work in a fast-paced, agile team environment.
• Prior experience with AI frameworks and engines such as TensorRT, PyTorch, ONNX, OpenVINO, vLLM, or TRT-LLM (preferred).
• Knowledge of GPU memory management, cache management, high-performance networking, or contributing to large open source projects (preferred).