Senior Product Manager, AI Inference Software

Amd

Confirmed live today High trust
Remote

Quick summary

Work type
Remote
Location
San Francisco, CAAustin, TX
Salary
$205,680–$308,520 / yr
Posted
118 days ago
Freshness
Confirmed live today
Closes
Jul 1, 2027

Market check

Salary context

Above market

How this pay compares to similar roles

Similar $201k
This role $257k
$149k $326k
below market most similar roles pay here above market

This role pays more than 89% of similar roles. Most pay $177,250–$224,500 — the blue band above. At the midpoint, this role pays about $257k versus about $201k for comparable roles.

Based on 240 similar postings.

Employer

About Amd

AMD (Advanced Micro Devices) is a semiconductor company that develops high-performance processors, graphics cards, and adaptive computing solutions for gaming, data centers, and embedded markets. Industry: Semiconductors

Amd currently has 496 open roles on FindRole.

Listed pay typically runs $172,000–$258,000 across 496 roles with salary data.

Most-posted roles

View all roles at Amd

At a glance

TL;DR · Senior Product Manager, AI Inference Software

The Senior Product Manager –AI Inference Software owns the framework-layer inference product strategy for ROCm. Working within the software ecosystem team, this role translates customer and engineering signals into roadmap decisions for production AI inference on AMD Instinct hardware. You will define how the framework-layer software stack enables production workloads from single-GPU to rack-scale deployments, focusing on performance, developer experience, and portability. Key responsibilities include partnering with engineering teams on inference engines, serving, orchestration, memory systems, and GPU libraries to identify performance bottlenecks. You will engage with the open-source community and sophisticated AI infrastructure customers to align technical requirements. Required expertise includes production AI inference systems like vLLM, SGLang, and AMD ATOM, along with serving systems like llm-d, distributed inference, and GPU computing.

What does a Product Manager earn in California?

Median $222500 from 327 postings across 57 companies.

See salary data

What you'll do

  • Own the product strategy and roadmap for ROCm’s inference framework software capabilities.
  • Define software stack requirements for production workloads from single-GPU to rack-scale deployments.
  • Translate customer and ecosystem needs into prioritized engineering requirements across organizational boundaries.
  • Identify inference performance bottlenecks and determine where framework-level software investments provide the most impact.
  • Provide software-informed input for future GPU architecture and platform decisions.
  • Serve as an active AMD presence in the open-source inference community to build relationships with maintainers and researchers.
  • Work directly with AI infrastructure customers to understand deployment requirements and translate signals into scalable product decisions.
  • Collaborate with cloud providers and model developers on joint integrations and solutions.

What we're looking for

  • Bachelor’s degree in Computer Science, Electrical Engineering, or a related technical field.
  • Advanced degree in a technical field (preferred).
  • Deep expertise in AI inference infrastructure, including inference engines, serving, orchestration, and memory management.
  • Experience driving complex technical initiatives across multiple engineering teams or organizations.
  • Experience working with sophisticated AI infrastructure customers, open-source communities, or ecosystem partners.
  • Experience in AI inference infrastructure, GPU computing, distributed systems, cloud infrastructure, high-performance computing, or technical product leadership.
  • Open-source contributions, technical writing, conference talks, or sustained community engagement (preferred).
  • Product management experience in deeply technical products (preferred).

More like this

Similar roles

Senior Product Manager, AI Platform Inference

Nvidia

Santa Clara, CA 83 days ago $168,000–$258,750
Generative AI Inference vLLM SGLang FlashInfer TensorRT-LLM Triton Dynamo TorchAO GPU Architecture Performance Profiling HW/SW Co-design Machine Learning Software Development SDKs Libraries GitHub
6+ yrs exp

Senior Product Manager, AI Inference Performance

Nvidia

Santa Clara, CA 5 days ago $208,000–$327,750
AI Inference TensorRT-LLM vLLM SGLang NVIDIA Dynamo Triton Inference Server Quantization KV Caching Speculative Decoding LLM Capacity Planning Autoscaling SLA Management Benchmarking
10+ yrs exp

Distinguished Engineer, Scaled Out Inferencing

Nvidia

Remote (Santa Clara, CA) 77 days ago $320,000–$488,750
GPU Architecture CUDA TensorRT-LLM vLLM SGLang Ray Kubernetes Linux Distributed Systems Model Serving Hardware Acceleration Cloud-Native Architecture High-Performance Computing Dynamo
10+ yrs exp Remote

Senior Product Manager, Inference Platform

Nvidia

Santa Clara, CA 29 days ago $208,000–$327,750
TensorRT-LLM vLLM Triton Inference Server MLOps AI/ML Systems Inference Serving Cloud Infrastructure KV Cache Management Speculative Decoding Continuous Batching Capacity Planning Resource Scheduling Model Serving Architecture
10+ yrs exp Hybrid

Senior Product Manager, AI

Brex

San Francisco, CA +2 45 days ago $184,000–$230,000
AI ML Evals Agentic Software Search and Ranking Risk and Fraud Detection Prototyping
Hybrid

Senior Product Manager, AI

Brex

San Francisco, CA +2 38 days ago $184,000–$230,000
AI ML Agentic Software Evals Search and Ranking Risk and Fraud Detection Prototyping
Hybrid