Senior Staff Engineer - Machine Learning Inference

What you'll do

• Architect and optimize Shopify’s production ML inference systems for high throughput and low latency.
• Extend technologies like CUDA, TensorRT, and Triton to deliver state-of-the-art performance at scale.
• Drive cost optimization by reducing cloud spend and carbon footprint without compromising model quality.
• Lead deep performance investigations using advanced techniques such as pruning, quantization, and batching.
• Set technical strategy for ML inference across Shopify, mentoring others in the field.
• Collaborate with cross-functional teams to deploy cutting-edge models powering the platform.

What we're looking for

• Proven expertise in optimizing large-scale ML inference systems with measurable performance improvements.
• Deep hands-on experience in production model serving using GPUs (CUDA, TensorRT) for high throughput and low latency.
• Strong software engineering skills in Python, C++, and relevant languages for distributed computing systems.
• Leadership in architecting reliable real-time inference systems handling millions of queries daily.
• Track record of cross-functional collaboration with ML research/engineering, infrastructure, and product teams.
• Advanced knowledge of model compression techniques, quantization, and efficient deployment strategies.