Deep Learning Software Engineer, TensorRT Performance

What you'll do

• Establish performance benchmarking methodologies and analysis workflows for the NVIDIA inference ecosystem.
• Contribute features and code to inference frameworks like TensorRT, TensorRT-LLM, and Torch-TensorRT.
• Develop optimized model pipelines for quantization, scheduling, memory management, and distributed inference.
• Implement graph compiler algorithms, frontend operators, and code generators across the inference ecosystem.
• Identify and optimize performance for state-of-the-art models across datacenter GPUs and edge SoCs.
• Scale deep learning model performance across various architectures and types of NVIDIA accelerators.
• Integrate TensorRT into open-source frameworks such as PyTorch and TensorRT-EdgeLLM.

What we're looking for

• Bachelor's, Master's, PhD, or equivalent experience in Computer Science, Computer Engineering, EECS, or AI.
• 2 years of relevant software development experience.
• Strong C++, Python programming, and software engineering skills.
• Experience with DL frameworks such as PyTorch, JAX, TensorFlow, or ONNX.
• Experience with inference libraries such as TensorRT, TensorRT-LLM, vLLM, SGLang, or FlashInfer.
• Experience with performance analysis and performance optimization.
• Strong foundation and architectural knowledge of GPUs (preferred).
• Deep understanding of modern deep learning models and workloads (preferred).
• Proficiency in a deep learning programming domain specific language like CUDA, TileIR, CuTeDSL, cutlass, or Triton (preferred).
• Prior contributions to major LLM inference frameworks or experience with graph compilers in deep learning inference (preferred).
• Prior experience optimizing performance for low-latency, resource-constrained systems or embedded AI pipelines (preferred).