Staff Software Engineer, AI Inference Cloud

Arm Holdings

Confirmed live today High trust

Hybrid

Quick summary

Work type: Hybrid
Location: Seattle, WA
Salary: $209,100–$282,900 / yr
Posted: 48 days ago
Freshness: Confirmed live today
Nearby: 99+ roles within 25 mi

Market check

Salary context

Above market

How this pay compares to similar roles

Similar $225k

This role $246k

$172k $295k

below market most similar roles pay here above market

This role pays more than 71% of similar roles. Most pay $195,950–$254,750 — the blue band above. At the midpoint, this role pays about $246k versus about $225k for comparable roles.

Based on 240 similar postings.

Employer

About Arm Holdings

Arm Holdings plc is a leading British semiconductor and software design firm, established in 1990 and recognized for developing energy-efficient processor architectures that power nearly all smartphones and a vast range of IoT and computing devices.

Arm Holdings currently has 113 open roles on FindRole.

Listed pay typically runs $208,450–$282,050 across 112 roles with salary data.

Most-posted roles

View all roles at Arm Holdings

At a glance

TL;DR · Staff Software Engineer, AI Inference Cloud

Role Posting Log in to save

As a Staff Software Engineer on the AI Inference Cloud team, you will shape the technical direction and develop highly available, scalable services for running AI inference workloads. You will define and build the architecture for cloud-based AI inference services, developing Kubernetes controllers and platform capabilities for workload deployment, scheduling, recovery, scaling, and lifecycle management. You will establish production practices for health validation, progressive rollout, and observability while improving platform reliability and resource efficiency. The role requires deep production experience with Kubernetes, networking, and compute infrastructure. You must possess strong software engineering skills in Go, C++, Rust, or Python. You will also work with frameworks like PyTorch, Ray, vLLM, SGLang, or TensorRT-LLM to solve complex problems regarding inference performance and accelerator-backed workloads.

What does a Software Engineer earn in Washington?

Median $202800 from 409 postings across 38 companies.

See salary data

Skills

Kubernetes Go C++ Rust Python Distributed Systems Cloud Platforms Observability PyTorch Ray vLLM SGLang TensorRT-LLM APIs Networking Capacity Planning

What you'll do

Define and build the architecture for cloud-based AI inference services.
Develop Kubernetes controllers for workload deployment, scheduling, recovery, and lifecycle management.
Establish production practices for health validation, progressive rollouts, and observability.
Improve platform reliability, scalability, performance, and resource efficiency.
Lead production readiness reviews and resolve complex issues across networking and compute infrastructure.
Turn operational bottlenecks and incidents into durable platform improvements.
Lead design and build reviews while mentoring other engineers.

What we're looking for

5+ years of experience building distributed systems, cloud platforms, or production infrastructure.
Deep production experience with Kubernetes, including controllers, operators, scheduling, networking, and workload lifecycle management.
Strong software engineering skills with hands-on programming in Go, C++, Rust, Python, or similar languages.
Experience with reliable services, APIs, concurrency, observability, deployment safety, capacity planning, and incident response.
Track record of leading complex technical initiatives including architecture, implementation, debugging, and mentoring.
Ability to troubleshoot complex systems and communicate clearly with diverse technical backgrounds.
Experience with AI infrastructure, model serving, or accelerator-backed workloads (preferred).
Familiarity with PyTorch, Ray, vLLM, SGLang, TensorRT-LLM, or experience tuning distributed workloads (preferred).

Save