Principal AI Software Engineer

What you'll do

• Prototype full system software to evaluate hardware/software co-designed capabilities for memory TCO reduction.
• Characterize and optimize Large Language Model (LLM) inference workloads focusing on KV Cache management across various memory tiers.
• Develop proof-of-concepts and evaluation frameworks to assess memory-tiering architectures for AI inference.
• Execute workload characterization studies for agentic, multi-turn, and long-context AI workloads to quantify performance metrics.
• Analyze end-to-end data movement across GPU, CPU, storage, and networking subsystems to identify optimization opportunities.
• Develop software prototypes and instrumentation to evaluate KV Cache offload, prefetching, and memory-overcommit techniques.
• Build performance models and simulation frameworks to predict the impact of memory hierarchy innovations on large-scale deployments.
• Influence hardware architecture and industry alignment through data-driven analysis and technical recommendations.

What we're looking for

• Bachelor's Degree in Computer Science or related technical field.
• 6+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python.
• Ability to pass Microsoft Cloud Background Check and other required security screenings.
• 12+ years of experience in systems software including OS kernel, memory management, I/O stacks, and virtualization (preferred).
• 10+ years of experience leading hardware/software co-design projects involving CPU or systems architecture (preferred).
• Deep expertise in Linux kernel internals, memory management, I/O subsystems, NUMA, DMA, and GPU/CPU/storage/network data paths (preferred).
• Hands-on experience with NVIDIA GPU software stacks including CUDA, NCCL, GDS, and GDR (preferred).
• Experience with AI inference infrastructure, large GPU clusters, and optimization of KV Cache intensive workloads (preferred).