ML Software Engineer

What you'll do

• Develop and maintain high-performance inference frameworks using Swift and C++ to distribute workloads across SoC hardware.
• Optimize ML inference performance by implementing techniques like quantization, sparsity, batching, and KV cache management.
• Coordinate and manage data movement across multi-node inference clusters for reliable service delivery.
• Integrate inference code into a full production service stack to ensure high reliability and scalability.
• Port the inference stack to new generations of SoCs and hardware acceleration IP.
• Implement new features based on emerging research from the machine learning community.
• Monitor system health using logs, metrics, and traces to resolve production incidents and maintain SLOs.
• Balance latency and throughput trade-offs through model-system co-design and systems-level optimizations.

What we're looking for

• Bachelor's degree in Computer Science, Computer Engineering, or a related field, or equivalent practical experience.
• At least 2 years of practical experience in the field.
• Experience building large-scale distributed systems for ML inference across processes, hosts, and service tiers.
• Proficiency in a systems or server language such as Python, Go, Rust, Java, or C++.
• Ability to reason about latency/throughput trade-offs and perform system-level optimizations for ML models.
• Experience with low-level systems programming, performance optimization, or hardware/SoC bring-up.
• Knowledge of server-side Swift, RPC/networking stacks (gRPC, Protocol Buffers), and concurrency handling.
• Expertise in ML inference serving optimizations including quantization, sparsity, batching, and GPU acceleration.