Staff AI Platform Engineer, Infrastructure Services

What you'll do

• Architect, harden, and scale the Kong AI Gateway infrastructure including authentication, rate limiting, and semantic caching.
• Lead reliability efforts by performing root-cause analysis and building monitoring systems for gateway performance issues.
• Design cross-platform solutions integrating CI/CD pipelines, GitOps workflows, and Kubernetes deployment tooling.
• Manage self-hosted LLM inference stacks including GPU capacity planning, autoscaling, and cost optimization.
• Establish LLMOps practices such as model versioning, evaluation, and infrastructure for retrieval-augmented generation.
• Evaluate and roll out AI developer tools while providing build-vs-buy recommendations to leadership.
• Track and report on token usage, latency, and costs across both API-based and self-hosted models.
• Set technical direction and mentor engineers to establish high standards for AI infrastructure.

What we're looking for

• 8 or more years of experience in platform, infrastructure, or DevOps engineering owning systems end-to-end in production.
• Hands-on experience with API gateway technologies (Kong, Envoy, Apigee) and LLM gateway patterns.
• Strong Kubernetes and GitOps experience using tools like ArgoCD across multiple environments.
• Solid CI/CD background including Jenkins pipeline design and GitHub Actions runner management.
• Experience with artifact management systems (Artifactory, Xray), source control administration, and infrastructure-as-code (Terraform).
• Experience deploying self-hosted LLM inference stacks (vLLM, NVIDIA Triton, TGI, Ollama) and GPU-backed infrastructure.
• Familiarity with LLMOps practices including model versioning, evaluation, and cost observability.
• Track record of setting technical direction, driving cross-team initiatives, and mentoring other engineers.
• Experience operating AI-assisted developer tooling at scale (preferred).
• Familiarity with Okta/OIDC and enterprise authentication patterns (preferred).
• Experience with engineering productivity metrics, AI code review tools, vector databases, or RAG pipelines (preferred).
• Exposure to model fine-tuning or lightweight training pipelines (preferred).