This role pays more than
71%
of similar roles. Most pay
$192,070–$254,750
— the shaded band above.
At the midpoint, this role pays about
$246k
versus about
$223k
for comparable roles.
Based on 240 similar postings.
Employer
About Arm Holdings
Arm Holdings plc is a leading British semiconductor and software design firm, established in 1990 and recognized for developing energy-efficient processor architectures that power nearly all smartphones and a vast range of IoT and computing devices.
Arm Holdings currently has
109 open roles
on FindRole.
Listed pay typically runs
$209,100–$282,900
across 108 roles with salary data.
Staff Software Engineer, AI Inference Runtime will join the AI Inference Runtime team to set technical direction for critical components of distributed inference runtimes for state-of-the-art models. The role involves leading hands-on work in scheduling, batching, KV-cache management, memory allocation, and kernel development while profiling system bottlenecks to optimize data-movement paths across compute, memory, and networking. You will define architectures for inference capabilities, enable new model architectures through operator support, and build benchmarking and validation systems to improve latency and throughput. Required expertise includes experience in ML systems, high-performance systems, compilers, or kernel development. Candidates should possess strong programming skills in C++, Rust, or Python, with deep knowledge of concurrency and parallel programming. The work focuses on the technical challenge of optimizing how efficiently new models utilize available compute resources within an AI platform.
What does a Software Engineer earn in Washington?
Median $202800 from 341 postings across 36 companies.
C++
Rust
Python
Kernel Development
ML Systems
Compiler
Assembly
performance-engineering
Parallel Programming
Memory Management
Collective Communication
Graph Optimization
Benchmarking