Software Engineer, ML Serving Platform

What you'll do

• Build self-serve tools, APIs, and workflows to help internal teams deploy and operate models independently.
• Integrate evolving open source inference frameworks and models into reliable production services at scale.
• Optimize systems for routing prediction requests, retrieving online features, and executing models on CPU and GPU infrastructure.
• Develop modular components for a disaggregated serving architecture to allow independent scaling of routing and execution.
• Improve Kubernetes deployment, autoscaling, and resource efficiency to meet latency and availability requirements.
• Create validation and migration tooling to simplify the adoption of new serving capabilities and global platforms.
• Use benchmarking, profiling, and tracing to identify bottlenecks and maintain production performance and reliability.

What we're looking for

• You must have 2+ years of software engineering experience building and maintaining production services or infrastructure.
• You must have strong computer science fundamentals and proficiency in Java, Kotlin, Go, C++, or Python.
• You must understand distributed systems fundamentals, including concurrency, networking, timeouts, failure handling, and performance trade-offs.
• You must have experience debugging production systems and using operational data to improve reliability, performance, or cost.
• You must hold a degree in Computer Science or a related field, or have equivalent practical experience.
• Experience with ML inference infrastructure, online feature retrieval, or other latency-sensitive distributed services (preferred).
• Experience operating containerized workloads on Kubernetes, including deployment, resource management, or autoscaling (preferred).
• Experience building self-serve developer platforms, APIs, or automation that helps other engineers move faster (preferred).