Distinguished Architect, AI

Datadog

Confirmed live today High trust
Hybrid

Quick summary

Work type
Hybrid
Location
New York, NYSan Francisco, CA
Salary
$330,000–$440,000 / yr
Posted
151 days ago
Freshness
Confirmed live today

Market check

Salary context

Above market

How this pay compares to similar roles

Similar $225k
This role $385k
$136k $473k
below market most similar roles pay here above market

This role pays more than 97% of similar roles. Most pay $194,316–$254,925 — the blue band above. At the midpoint, this role pays about $385k versus about $225k for comparable roles.

Based on 240 similar postings.

Employer

About Datadog

Datadog, Inc. is an American company that provides an observability service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform.

Datadog currently has 73 open roles on FindRole.

Listed pay typically runs $192,000–$240,000 across 71 roles with salary data.

Most-posted roles

View all roles at Datadog

At a glance

TL;DR · Distinguished Architect, AI

The Distinguished Architect, AI joins the Product Solutions Architecture team to serve as a technical multiplier for leading AI labs and AI-native companies. This role involves bridging infrastructure aspirations with the technology roadmap to solve observability challenges for training and deploying foundational models at scale. Key responsibilities include providing strategic advisory to founders and research leads, leading deep-dive architecture reviews, and co-creating custom observability integrations. The successful candidate will demonstrate thought leadership in the AI/LLM space while identifying emerging technology shifts for product management. Required expertise includes at-scale distributed systems architecture, high-performance computing, accelerator hardware like GPUs and TPUs, and modern orchestration frameworks. The role focuses on solving complex LLM Ops and LLM Observability problems within high-throughput hyper-scale AI workloads.

What does a Architect earn?

Median $225000 from 45 postings across 26 companies.

See salary data

What you'll do

  • Act as a technical multiplier for leading AI labs and AI-native companies.
  • Guide founders and infrastructure leads on best practices and emerging trends in the AI/LLM space.
  • Lead deep-dive architecture reviews and design engagements for high-throughput hyper-scale AI workloads.
  • Feed emerging AI-native technology shifts directly back to Datadog Product Management.
  • Co-create custom observability integrations and solutions alongside Product Solutions Architects.
  • Provide high-quality technical resources to a broad audience of practitioners and economic buyers.
  • Assist leadership in recruiting and hiring top talent for the Product Solutions Architecture and Field CTO teams.

What we're looking for

  • 10+ years of experience with at-scale distributed systems architecture, high-performance computing, or large-scale infrastructure.
  • Deep familiarity with the AI/LLM ecosystem, accelerator hardware (GPUs/TPUs), and modern orchestration frameworks.
  • Proven experience interacting with and influencing research scientists, technical founders, and elite individual contributors.
  • Strong understanding of best practices and real-world challenges in AI/LLM Ops and LLM Observability.
  • Excellent customer-facing presentation skills for large audiences and executives.
  • Ability to link product functionality to business objectives, value realization, and ROI.
  • Ability to travel via auto, train, or air up to 50% of the time.
  • Bachelor’s degree in engineering or related field; Master’s degree (preferred).

More like this

Similar roles

Distinguished Architect

SentinelOne

Remote 148 days ago $248,400–$341,550
Enterprise Architecture AI API Design Identity Integration Security-by-Design Data Classification Technology Radar Architecture Decision Record (ADR) Architecture Review Board (ARB) Enterprise Applications
Remote

AI Architect

Salesforce

San Francisco, CA 47 days ago
Python AWS LLMs Generative AI MLOps NLP Deep Learning Apex Agentforce Data Cloud Prompt Engineering Machine Learning Cloud Security Data Preprocessing Feature Engineering Data Synthesis
10+ yrs exp Hybrid

Distinguished AI Engineer

Capital One Financial

San Francisco, CA +4 56 days ago
Python Go Scala Java C++ C# PyTorch Huggingface AWS Google Cloud Azure VectorDBs Nemo Guardrails LLM Inference Similarity Search Machine Learning
8+ yrs exp

Distinguished AI Engineer

Elevance Health

Indianapolis, IN +8 12 days ago $254,584–$416,592
Large Language Models (LLMs) Agentic Systems Distributed Systems Python Java Microservices Event-Driven Architecture APIs Cloud Platforms CI/CD DevOps Software Development Lifecycle (SDLC)
10+ yrs exp Hybrid

Distinguished Technologist, AI Model Performance Architect

HP Inc.

Spring, TX +1 139 days ago $190,000–$274,000
AI Model Performance HW/SW Co-design Hardware Architecture Electrical Engineering FPGA MATLAB Systems Design Simulations Schematic Capture Printed Circuit Board GPU NPU CPU Memory Architecture Debugging New Product Development
10+ yrs exp