Principal Software Engineer, AI Foundations

What you'll do

• Design and operate a high-throughput, low-latency LLM serving platform across GPU and CPU fleets.
• Build a GenAI Gateway and inference API layer featuring authentication, rate limiting, and standardized observability.
• Develop intelligent model routing to balance quality, latency, and cost across various model backends.
• Optimize GPU performance through kernel-level tuning, model compilation, and efficient memory management.
• Implement quantization and compression strategies to reduce costs and latency while preserving model quality.
• Design disaggregated serving patterns and distributed inference architectures to improve hardware utilization.
• Establish SLOs and operational excellence through load testing, capacity planning, and automated performance benchmarking.
• Architect agentic AI-enabled engineering workflows to improve code quality and delivery speed at scale.

What we're looking for

• Formal training or certification in software engineering concepts with 7+ years of applied experience in system design and application development.
• Expert proficiency in one or more programming languages such as Python, Java, Scala, or Go.
• Proven experience designing and operating high-scale inference and distributed systems including multi-tenant services and rate limiting.
• Strong understanding of GPU-serving fundamentals, including compute/memory trade-offs, batching, and performance profiling.
• Experience with model serving stacks, rollout strategies, and cloud-native technologies like containers and orchestration.
• Demonstrated experience leading the adoption of agentic AI-enabled development practices and responsible AI governance.
• Strong communication skills to influence senior leaders and translate complex technical topics into clear decisions.
• Deep experience with LLM inference optimization, quantization, and disaggregated serving architectures (preferred).