Senior AI Platform Engineer, Infrastructure Services

What you'll do

• Architect, harden, and scale the Kong AI Gateway infrastructure including authentication, rate limiting, and semantic caching.
• Lead reliability efforts by performing root-cause analysis and building monitoring systems for gateway performance issues.
• Design cross-platform solutions integrating CI/CD pipelines, GitOps workflows, and Kubernetes deployment tooling.
• Manage self-hosted LLM inference stacks including GPU capacity planning, autoscaling, and cost optimization.
• Develop LLMOps practices such as model versioning, evaluation, and infrastructure for retrieval-augmented generation.
• Evaluate and roll out AI developer tools while providing build-vs-buy recommendations to leadership.
• Track and report on token usage, latency, and costs across both API-based and self-hosted models.
• Set technical direction and mentor other engineers on AI infrastructure standards and architecture.

What we're looking for

• 5 or more years of experience in platform, infrastructure, or DevOps engineering owning systems end-to-end in production.
• Hands-on experience with API gateway technologies (Kong, Envoy, Apigee) and LLM gateway patterns.
• Strong Kubernetes and GitOps experience using tools like ArgoCD across multiple environments.
• Solid CI/CD background including Jenkins pipeline design and GitHub Actions runner management.
• Experience with artifact management systems (Artifactory, Xray) and GitHub Enterprise administration.
• Working knowledge of infrastructure-as-code (Terraform) and cloud platforms (AWS/EKS).
• Experience deploying self-hosted LLM inference stacks (vLLM, NVIDIA Triton, TGI, Ollama) and GPU-backed infrastructure.
• Familiarity with LLMOps practices, including model versioning, evaluation, and usage/cost observability.
• Experience operating LLM/AI-assisted developer tooling at scale in an enterprise (preferred).
• Familiarity with Okta/OIDC and enterprise authentication patterns (preferred).
• Experience with engineering productivity metrics tools and AI-based code review tools (preferred).
• Experience with vector databases, RAG pipelines, or model fine-tuning (preferred).