Senior Software Engineer, Machine Learning Platform

What you'll do

• Design and operate scalable machine learning and AI infrastructure on AWS.
• Build shared platform capabilities for LLM and agentic workloads including model access and workflow orchestration.
• Develop evaluation frameworks for non-deterministic AI systems using offline benchmarks and human feedback.
• Establish observability, reliability, and governance for models covering latency, cost, and safety.
• Build distributed training, batch inference, and large-scale processing systems using Ray or Spark.
• Maintain infrastructure as code using Terraform to support platform components.
• Develop data ingestion and streaming systems using technologies like Kinesis, Kafka, or Flink.
• Improve CI/CD workflows for machine learning models and AI applications.

What we're looking for

• 5+ years of experience in ML or AI infrastructure, platform engineering, distributed systems, or production ML systems.
• Experience designing distributed systems and large-scale data or compute platforms on AWS using Spark or Ray.
• Knowledge of the machine learning development lifecycle including preprocessing, training, evaluation, deployment, and monitoring.
• Working knowledge of LLM application patterns such as RAG, structured outputs, tool calling, and agent orchestration.
• Experience designing production systems integrating ML or foundation models through reliable APIs, workflows, and data contracts.
• Hands-on experience with CI/CD pipelines, DevOps practices, and infrastructure as code using Terraform.
• Experience with containerization and orchestration technologies such as Docker and Kubernetes.
• Strong programming skills in Python, Go, Scala, Java, or similar languages.
• Experience shipping LLM-powered or agentic systems to production (preferred).
• Experience with model gateways, prompt lifecycle management, retrieval/vector search, and agent orchestration frameworks (preferred).
• Experience building evaluation, tracing, and observability for non-deterministic AI systems (preferred).
• Familiarity with managed or self-hosted foundation model infrastructure like Amazon Bedrock or SageMaker (preferred).
• Experience operating GPU-based workloads and optimizing training/inference performance; CUDA experience is a plus (preferred).