Lead Infrastructure Developer

What you'll do

• Manage and own GPU fleet operations across the company's AWS estate.
• Build a custom scheduling layer for GPU resources from the ground up.
• Diagnose and resolve GPU and node failures by coordinating with AWS support and capacity channels.
• Provide researcher support through office hours, dedicated support channels, and self-service tooling.
• Instrument the fleet to track utilization, queue depth, job success rates, and cost metrics.
• Manage Infrastructure as Code (IaC) setups, permissions, and technical infrastructure architecture.
• Act as the primary technical contact for external compute partners and internal infrastructure teams.
• Author technical documentation including runbooks, decision records, and onboarding guides.

What we're looking for

• 8+ years of experience operating production infrastructure with deep hands-on AWS expertise in EC2 GPU fleets, EKS, IAM, VPC, S3, and FSx.
• Experience scheduling, diagnosing, and managing GPUs specifically within the AWS environment.
• Experience operating GPU fleets at a scale of 1000+ GPUs.
• Expertise in scripting and automation using Python, PowerShell, bash, or equivalent.
• Expertise in infrastructure as code using Terraform or equivalent.
• Familiarity with a GPU scheduling or orchestration layer such as Slurm, Kubernetes (with Kueue or Volcano), Ray, dStack, or SkyPilot.
• Observability practice including Grafana, Prometheus, or equivalent tools.