Software Engineer, Machine Learning Platform - Gen AI

What you'll do

• Build production infrastructure for real-time GPU serving, high-throughput batch inference, and model fine-tuning.
• Design scalable systems for GPU autoscaling, utilization, and backend services.
• Develop and maintain the LLM Gateway, Agent Gateway, evals infrastructure, and guardrails.
• Optimize GPU inference to reduce costs and latency for open-weight and closed-source models.
• Build platforms that support rapid experimentation while meeting production standards for monitoring and SLOs.
• Implement post-training techniques such as RLHF, RLVR, and agent optimization.
• Develop cost attribution and observability tools for centralized GenAI platform services.

What we're looking for

• B.S., M.S., or PhD. in Computer Science or equivalent.
• 3+ years of industry experience in software engineering.
• Strong backend engineering fundamentals, especially in Python and distributed systems.
• Experience building production services, APIs, data pipelines, or ML infrastructure at scale.
• Experience operating systems in production, including observability, debugging, reliability, incident response, and performance/cost optimization.
• Hands-on experience with LLM inference and/or fine-tuning of open-weight models in production.
• Proficiency in using AI coding tools in the full software development lifecycle.
• Experience with LLM inference engines, distributed fine-tuning, or GPU performance work (preferred).