Intern AI Application Engineer

What you'll do

• Deploy and operate an LLM inference service on a 4x NVIDIA H200 GPU server.
• Build an automated benchmark harness to replay request traces and record latency and throughput metrics.
• Configure KV cache offloading to network-attached storage over NVMe-oF and RDMA.
• Run daily experiment sweeps across request rates, context lengths, and cache sizes.
• Analyze performance bottlenecks across the GPU, network, and SSD using Python.
• Debug the end-to-end data path and validate KV cache correctness with hardware engineers.
• Summarize measured storage requirements to shape the architecture of future SSD controllers.

What we're looking for

• Must have at least 1 academic quarter/semester remaining.
• Pursuing a Bachelor's, Master's, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related field (preferred).
• Hands-on experience running LLM inference with Python and PyTorch on NVIDIA GPUs under Linux using frameworks like vLLM, SGLang, or Hugging Face TGI.
• Understanding of LLM serving internals including KV cache, prefix caching, batching, parallelism, and latency metrics like TTFT and TPOT.
• Experience benchmarking and profiling system performance and analyzing results with scripts such as Python, pandas, and plotting.
• Familiarity with KV cache offloading, RDMA/RoCE, NVMe/NVMe-oF storage, and containers like Docker or Kubernetes (preferred).
• Daily onsite presence at the San Jose, CA headquarters is required.