ML Software Engineer

What you'll do

• Engineer continuous improvements in stability and performance for the Private Cloud Compute ML-inference stack.
• Implement new functionalities as they emerge from the research community.
• Bring the inference stack up on new generations of SoCs and hardware acceleration IP.
• Write performant frameworks in Swift and C++ to distribute ML inference across SoC acceleration blocks.
• Coordinate work and move data reliably across multi-node inference clusters.
• Integrate inference code into a full service stack to serve user traffic reliably and performantly.
• Optimize ML inference serving through quantization, sparsity, batching, and KV cache management.
• Drive production incidents to root cause using logs, metrics, and traces.

What we're looking for

• Bachelor's degree in Computer Science, Computer Engineering, or a related field, or equivalent practical experience.
• 2 years of practical experience.
• Experience building large-scale distributed systems that serve ML inference across processes, hosts, and service tiers.
• Ability to reason about latency/throughput trade-offs and distinguish system-level solutions from model-system co-design.
• Proficiency in a systems or server language such as Python, Go, Rust, Java, or C++.
• Low-level systems programming, performance, or hardware/SoC bring-up experience (preferred).
• Experience with server-side Swift, RPC/networking stacks, or ML inference serving optimizations (preferred).
• Strong debugging, observability, and concurrency skills, including experience with Apple platforms (preferred).