Senior Software Engineer – TensorRT Edge-LLM

What you'll do

• Develop and evolve a state-of-the-art inference framework for autoregressive models.
• Design compiler and runtime optimizations for transformer-based models on constrained platforms.
• Contribute to CUDA kernel development for critical transformer components like attention and GEMM.
• Benchmark and optimize inference performance across diverse embedded environments.
• Stay updated with emerging techniques in the LLM/VLM ecosystem and integrate them into software.

What we're looking for

• 4+ years of relevant software development experience in AI or related field.
• Deep understanding of transformer models and inference optimization techniques.
• Proficient programming ability with modern C++ (C++11/14/17 and beyond).
• Familiarity with TensorRT, vLLM, SGLang, MLC-LLM, FlashInfer frameworks/libraries.
• Experience in CUDA kernel development, performance profiling, GPU architecture.
• Track record of strong software design, execution, and cross-team collaboration.
• Prior work on autoregressive LLM serving systems including speculative decoding.