Senior Software Engineer, Distributed Systems Engineering

What you'll do

• Develop custom software to schedule GPU resources on Kubernetes clusters.
• Implement monitoring and health management capabilities for high-availability GPU assets.
• Analyze multiple data streams including hardware diagnostics, cluster telemetry, and network data.
• Ensure production AI clusters run reliably with maximum performance across various workloads.
• Evaluate system failures and improve services through a defined incident management process.
• Develop software using Kubernetes APIs and frameworks rather than just operating the cluster.
• Automate and manage large-scale distributed systems independent of cloud providers.

What we're looking for

• Experience in a software engineering role within a highly technical organization with demonstrable impact.
• 8+ years of experience in a similar role and on large-scale production systems.
• Software development experience with Kubernetes APIs and frameworks, not just operating a cluster.
• Proficiency in a systems programming language such as Go or Python.
• Solid understanding of data structures and algorithms.
• A BS in Computer Science, Engineering, Physics, Mathematics, or a comparable degree or equivalent experience.
• Technical competency in managing and automating large-scale distributed systems independent of cloud providers (preferred).
• Advanced hands-on experience with cluster management systems like Kubernetes, Slurm, or Bright Cluster Manager (preferred).