Principal Engineer, AI Serving Framework Architect (Software)

Samsung Semiconductor

Quick summary

Work type
On-site
Location
San Jose, CA
Salary
$219,000–$351,000 / yr
Posted
today

Market check

Salary context

Above market

How this pay compares to similar roles

Similar $209k
This role $285k
$151k most similar roles pay here $372k

This role pays more than 89% of similar roles. Most pay $172,500–$246,150 — the shaded band above. At the midpoint, this role pays about $285k versus about $209k for comparable roles.

Based on 240 similar postings.

Employer

About Samsung Semiconductor

Samsung Semiconductor is the global semiconductor business unit of Samsung Electronics, designing and manufacturing memory chips, logic semiconductors, and foundry solutions for a broad range of applications.

Samsung Semiconductor currently has 54 open roles on FindRole.

Listed pay typically runs $163,000–$253,000 across 54 roles with salary data.

Most-posted roles

View all roles at Samsung Semiconductor

At a glance

TL;DR · Principal Engineer, AI Serving Framework Architect (Software)

As a Principal AI System Architect at Samsung’s Architecture Research Lab, you will lead research teams in Korea and drive technical direction for next-generation AI system architectures. Your primary responsibilities include developing performance models for multi-rack scale memory-centric systems, researching dynamic scheduling methodologies to maximize AI inference performance, and investigating methods to accelerate search operations using compute-capable memory. You will also propose software designs for implementing optimization algorithms on open-source platforms like vLLM. The ideal candidate has a PhD in Computer Science with over 10 years of experience in AI Serving Frameworks, extensive knowledge of PyTorch, Python, and C++, and expertise in profiling and optimizing AI inference systems. This role requires a deep understanding of compute, memory, and networking bottlenecks in large-scale AI systems, as well as strong collaboration skills to tackle complex challenges in the rapidly evolving field of AI hardware and software integration.

What you'll do

  • Lead research teams in Korea and propose technical direction for AI serving frameworks.
  • Research dynamic scheduling methodologies to maximize AI inference performance in multi-rack systems.
  • Investigate methods to accelerate search operations using compute-capable memory in hierarchical setups.
  • Study optimal placement strategies for KVCache and vector DBs to minimize SSD accesses and IO stalls.
  • Propose software designs implementing optimization algorithms on open-source platforms like vLLM.

What we're looking for

  • PhD in Computer Science or related field with 10+ years of AI Serving Framework experience.
  • Led a project to build and optimize an LLM Inference Software Stack for multi-rack systems.
  • Extensive experience designing AI Inference Software Stacks for heterogeneous devices.
  • Proficiency in PyTorch, Python, C++, and AI inference system profiling and optimization.
  • Strong understanding of compute, memory, and networking bottlenecks in AI systems.
  • Expertise in dynamic scheduling methodologies for multi-rack scale memory-centric systems.

More like this

Similar roles

Principal Engineer, AI System Architect (Hardware)

Samsung Semiconductor

San Jose, CA today $219,000$351,000
Python C++ PyTorch LLMs DLRMs AI system hardware architectures system-level architectural research performance-per-watt metrics architecture requirements and trade-offs high-performance interconnects memory hierarchies cross-functional collaboration quantitative modeling

Principal Engineer, AI System Architect (Hardware)

Samsung Semiconductor

San Jose, CA today $219,000$351,000
Python C++ PyTorch AI system hardware architectures LLMs DLRMs performance-per-watt metrics event-driven simulation models system-level architectural research architecture-level design decisions high-performance interconnects memory hierarchies

Staff Engineer, AI System Architect (Hardware)

Samsung Semiconductor

San Jose, CA today $163,000$253,000
Python C++ PyTorch AI LLMs DLRMs system-level architectural research event-driven simulation models high-performance interconnects memory hierarchies performance evaluation design-space exploration communication skills collaboration

AI/ML Platform Architect - Engineer, Principal

Qualcomm

San Diego, CA 100 days ago $200,800$301,200
Python C++ TensorFlow PyTorch GPU NPU Windows Agentic AI Computer Vision Audio Generative AI Version Control Systems Agile Project Management High Performance Computing Profiling Tools Software Development Methodologies Embedded Systems Computer Architecture