Senior Software Engineer, Inference Platform

MongoDB

Hybrid

Quick summary

Work type: Hybrid
Location: New York, NYPalo Alto, CA
Salary: $126,000–$248,000 / yr
Posted: 13 days ago
Nearby: 99+ roles within 25 mi

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $185k

This role $187k

$111k most similar roles pay here $263k

This role pays more than 50% of similar roles. Most pay $147,568–$222,000 — the shaded band above. At the midpoint, this role pays about $187k versus about $185k for comparable roles.

Based on 240 similar postings.

Employer

About MongoDB

MongoDB is a leading American software company that develops and provides commercial support for a popular, source-available document database. Designed to handle unstructured and structured data natively, its platform is purpose-built for modern cloud applications, analytics, and AI experiences.

MongoDB currently has 287 open roles on FindRole.

Listed pay typically runs $126,500–$209,000 across 104 roles with salary data.

Most-posted roles

View all roles at MongoDB

At a glance

TL;DR · Senior Software Engineer, Inference Platform

Apply Now Log in to save

Join our Search and AI Platform team as a Senior Engineer to build the next-generation inference platform for MongoDB Atlas, supporting semantic search and hybrid retrieval. You will design and develop core systems that enable real-time, low-latency model inference at scale, collaborating with ML researchers from Voyage.ai to integrate embedding models and rerankers into production environments. Key responsibilities include enhancing latency-aware routing, health monitoring, and observability features while ensuring the platform’s performance and reliability in a cloud-native setting. Ideal candidates have 5+ years of experience building scalable backend systems using Go, Rust, Python, or C++, with expertise in distributed systems, multi-tenant service design, and vector search technologies like Faiss or HNSW.

Skills

Kubernetes Python Go Rust C++ AWS Azure GCP Docker CI/CD Prometheus Grafana Faiss HNSW ScaNN vLLM MongoDB Atlas Vector search systems Cloud-native architectures MLOps

What you'll do

Design and build components of a multi-tenant inference platform integrated with MongoDB Atlas.
Collaborate on productionizing embedding models for both batch and real-time use cases.
Enhance platform capabilities including latency-aware routing, health monitoring, and observability.
Improve performance and resource efficiency in cloud-native environments using Kubernetes.
Work across teams to ensure the inference platform meets scale, reliability, and latency demands.
Gain hands-on experience with tools like vLLM for model serving and inference.

What we're looking for

5+ years of experience building backend or infrastructure systems at scale.
Strong software engineering skills in Go, Rust, Python, or C++ with focus on performance and reliability.
Experience in cloud-native architectures, distributed systems, and multi-tenant service design.
Familiarity with ML model serving and inference runtimes, vector search systems preferred.
Comfortable collaborating across functional teams including ML researchers and backend engineers.
Motivated to work on systems integrated into MongoDB Atlas used by thousands of developers.

Similar roles

Senior Software Engineer, Machine Learning Inference

Nvidia

Santa Clara, CA 67 days ago $152,000–$241,500

C++ Python CUDA Rust TensorRT TensorRT-LLM vLLM SGLang PyTorch JAX Deep Learning Frameworks GPU Programming Performance Analysis Optimization Techniques CI/CD

Hybrid

Save

Senior Software Engineer, AI Inference Systems

Nvidia

Santa Clara, CA 49 days ago $184,000–$287,500

Python C/C++ CUDA Kubernetes Docker Triton PyTorch vLLM SGLang MLIR Linux Go Rust CI/CD AWS GCP Azure Prometheus Grafana GitHub MLOps

Hybrid

Save

Senior Software Engineer - AI Inference

Nvidia

Remote (Santa Clara, CA) 63 days ago $152,000–$241,500

Python C++ CUDA vLLM SGLang PyTorch Triton NCCL Dynamo CI/CD GPU InfiniBand Profiling Flamegraphs Microbenchmarks Concurrency Multi-threading Multi-process Kubernetes Docker PostgreSQL

Remote

Save