Senior Product Manager, AI Inference Performance

What you'll do

• Own the inference performance roadmap across the entire stack, including model representation, memory management, and request scheduling.
• Decide which optimization techniques to build, adopt, or retire to ensure high performance for diverse customer sizes.
• Develop platforms and capabilities that generalize across different model families and deployment topologies.
• Define the performance strategy for agentic applications, focusing on cross-turn cache reuse and request prioritization.
• Manage the integration of optimizations across frameworks like TensorRT-LLM, vLLM, SGLang, and NVIDIA Dynamo.
• Define and own the methodology for benchmarking performance metrics like TTFT, throughput, and cost per million tokens.
• Manage day-to-day product operations, including release readiness, quality bars, regression tracking, and customer feedback loops.

What we're looking for

• 12+ years of experience in product management at a technology company, or comparable time as a founder, engineering lead, or technical product owner.
• Depth in AI inference optimization including KV caching, quantization, speculative decoding, and disaggregated serving.
• Familiarity with inference and orchestration frameworks such as TensorRT-LLM, vLLM, SGLang, and NVIDIA Dynamo.
• Proven track record of working independently to define strategies and drive ambiguous problems to shipped outcomes.
• Operational experience running a live product including release management, quality rigor, and customer support processes.
• Ability to translate low-level technical capabilities into business value for both engineers and executives.
• BS, MS, or PhD in Computer Science, Computer Engineering, or another relevant area of study (or equivalent experience).
• Engineering experience with LLM inference performance, open-source contributions, or production experience at scale (preferred).