Senior Engineer, Competitive Benchmarking & Performance

What you'll do

• Own prioritized competitive benchmarking initiatives across search, vector, and observability workloads.
• Build and maintain automated benchmark harnesses, datasets, and load profiles reflecting real-world usage.
• Design fair comparisons using documented system versions, configurations, and resource budgets.
• Evaluate latency distributions, throughput, resource consumption, and cost efficiency for various workloads.
• Adapt and extend tools such as Elasticsearch Rally and ClickBench for performance testing.
• Investigate performance differences using profiling, system metrics, and controlled experiments.
• Evaluate self-managed and managed cloud deployments, including serverless scaling behavior.
• Publish reproducible benchmark artifacts and technical reports explaining methodology and findings.

What we're looking for

• Solid software engineering skills with experience building maintainable automation and analysis tools in Python or another relevant language.
• Hands-on experience evaluating and diagnosing performance in distributed systems, databases, or search infrastructure.
• Depth in at least one relevant area such as search, vector retrieval, analytical databases, or observability systems.
• Experience designing representative datasets and workloads, including query patterns, concurrency, and ingestion behavior.
• Sound experimental judgment including repeated measurements, variability analysis, and distinguishing meaningful changes from measurement noise.
• Experience building reproducible test environments and documenting configurations, procedures, and results.
• Clear technical writing and communication skills to explain complex findings and their limitations.
• Experience with Elasticsearch, Lucene, Rally, ClickBench, or vector search benchmark frameworks (preferred).