Engineering Manager, Inference Benchmarking

What you'll do

• Drive the technical roadmap for AIPerf's core infrastructure including load generation and ZMQ-based microservices.
• Manage GPU telemetry using DCGM, PyNVML, Prometheus metrics, and Kubernetes-native deployment.
• Ensure the accuracy and statistical soundness of benchmark results for industry-wide production infrastructure decisions.
• Advise on upstream engine integrations for vLLM, TRT-LLM, and SGLang in partnership with internal teams.
• Hire, mentor, and grow a team of senior engineers in a high-velocity open-source environment.
• Maintain AIPerf's relevance across emerging hardware, workload categories, and inference configurations.

What we're looking for

• Bachelor's degree in Computer Science, Electrical Engineering, or related field, or equivalent experience.
• Software engineering experience building performance-critical infrastructure, ML tooling, or distributed systems.
• 3+ years of engineering leadership experience as a tech lead, TLM, or engineering manager.
• Deep understanding of LLM inference mechanics including TTFT, ITL, KV caching, Prefill/Decode, and speculative decoding.
• Proven track record of collaborating across multi-functional groups and delivering production-quality output in high-velocity environments.
• Extensive experience with vLLM, TRT-LLM, or SGLang internals along with contributions to their upstream projects (preferred).
• Experience building Kubernetes-native infrastructure including operators, Helm charts, and GPU observability tooling (preferred).
• Background in competitive benchmarking frameworks such as MLPerf or equivalent industry-standard evaluation systems (preferred).