This listing doesn't post a salary. Most similar roles pay
$174,600–$228,062.
Based on 240 similar postings.
Employer
About JPMorgan Chase
JPMorgan Chase & Co. is a global financial services firm and one of the largest banks in the world, offering investment banking, commercial banking, asset management, and consumer financial services.
JPMorgan Chase currently has
1117 open roles
on FindRole.
Listed pay typically runs
$186,160–$215,000
across 7 roles with salary data.
As a Principal Software Engineer within the Chief Data and Analytics Office, you will join an agile team to build and evolve the firm’s GenAI serving platform. You will be responsible for designing high-throughput, low-latency LLM inference systems featuring intelligent model routing, GPU efficiency, and disaggregated serving patterns. Your daily work involves developing a GenAI Gateway with robust authentication and rate limiting while implementing quantization and compression strategies to optimize performance. You will manage infrastructure across GPU/CPU fleets, establish operational SLOs, and architect agentic AI-enabled engineering workflows. The role requires expert proficiency in Python, Java, Scala, or Go, alongside deep experience in distributed systems, cloud-native technologies, and ML system engineering. You will solve complex technical challenges regarding model routing, performance benchmarking, and secure, scalable inference to deliver reliable enterprise-scale AI capabilities across various firm portfolios.
What does a Software Engineer earn?
Median $197500 from 2196 postings across 133 companies.
Design and operate a high-throughput, low-latency LLM serving platform across GPU and CPU fleets.
Build a GenAI Gateway to manage authentication, rate limiting, and standardized observability for diverse workloads.
Develop intelligent model routing systems to balance quality, latency, and cost across multiple backends.
Optimize GPU performance through kernel tuning, model compilation, and efficient memory management strategies.
Implement quantization and compression techniques to reduce costs while maintaining high model quality.
Architect disaggregated serving patterns and distributed inference architectures to improve utilization and tail latency.
Establish SLOs/SLAs for inference services and build robust performance and cost observability dashboards.
Design agentic AI-enabled engineering workflows to automate development tasks like PR reviews and test generation.
What we're looking for
Formal training or certification in software engineering concepts with 7+ years of applied experience in system design and large-scale platform delivery.
Expert proficiency in at least one programming language such as Python, Java, Scala, or Go.
Proven experience designing and operating high-scale inference systems and distributed architectures including multi-tenancy and load shedding.
Strong understanding of GPU-serving fundamentals, including compute/memory trade-offs, batching, and performance profiling.
Experience with model serving stacks, rollout strategies, and performance benchmarking methodologies.
Practical experience with cloud-native technologies including containers, orchestration, infrastructure as code, and observability.
Demonstrated ability to lead agentic AI-enabled development practices while ensuring security, auditability, and risk-based governance.
Strong communication skills to influence senior leaders by translating complex technical topics into clear business decisions.