Principal AI Inference Systems Engineer

What you'll do

• Design and deliver reference architectures for LLM training and inference on AMD GPUs.
• Architect and validate Kubernetes-based distributed training stacks for large-scale LLM workloads.
• Define and implement gang scheduling and topology-aware GPU placement for multi-node training.
• Enable Kubernetes-native training controllers including Kubeflow Training Operator, MPI Operator, Volcano, and Kueue.
• Implement and validate GPU orchestration using Kubernetes GPU Operator, device plugins, metrics exporters.

What we're looking for

• Extensive experience in deploying and operating large-scale GPU clusters for production AI training and inference.
• Deep expertise in Kubernetes GPU orchestration including operators, device plugins, scheduling, multi-tenancy, and observability.
• Hands-on experience with distributed training on Kubernetes using Kubeflow, MPI Operator, Volcano, Kueue, and Ray.
• Strong knowledge of gang scheduling, elastic jobs, quotas, priority, and shared GPU environments in AI workloads.
• Tuned Kubernetes networking and storage for high-performance AI workloads including RDMA and scalable checkpointing.