Principal Software Engineer, Large-Scale LLM Memory and Storage Systems

What you'll do

• Design and evolve a unified memory layer spanning GPU, host, RDMA, and remote storage tiers.
• Architect and implement deep integrations with LLM serving engines like vLLM and TensorRT-LLM.
• Co-design interfaces for disaggregated prefill and peer-to-peer KV-cache sharing across heterogeneous clusters.
• Partner with hardware teams to exploit GPUDirect, RDMA, and NVLink for low-latency memory access.
• Set the technical direction and roadmap for memory and storage subsystems.
• Mentor senior and junior engineers on systems engineering and memory management.
• Represent the team in internal reviews and external forums like open-source projects and conferences.

What we're looking for

• Master's or PhD degree or equivalent experience.
• 15+ years of experience building large-scale distributed systems, high-performance storage, or ML systems infrastructure.
• Proficiency in C/C++ and Python.
• Deep understanding of memory hierarchies including GPU HBM, host DRAM, SSD, and remote/object storage.
• Experience with distributed caching or key-value systems optimized for low latency and high concurrency.
• Hands-on experience with networked I/O and RDMA/NVMe-oF/NVLink-style technologies.
• Strong skills in profiling and optimizing systems across CPU, GPU, memory, and network.
• Prior contributions to open-source LLM serving or systems projects focused on KV-cache optimization (preferred).