Agentic AI Test Engineer

Comcast

Quick summary

Work type: On-site
Location: Mount Laurel, NJ
Salary: $114,985–$172,478 / yr
Posted: 1 day ago

Market check

Salary context

Below market

How this pay compares to similar roles

Similar $203k

This role $144k

$100k most similar roles pay here $259k

This role pays less than 85% of similar roles. Most pay $162,000–$244,012 — the shaded band above. At the midpoint, this role pays about $144k versus about $203k for comparable roles.

Based on 240 similar postings.

Employer

About Comcast

Comcast is an American telecommunications and media conglomerate, providing cable TV, internet, and phone services under the Xfinity brand, and owning NBCUniversal.

Comcast currently has 42 open roles on FindRole.

Listed pay typically runs $129,515–$195,779 across 22 roles with salary data.

Most-posted roles

View all roles at Comcast

At a glance

TL;DR · Agentic AI Test Engineer

Apply Now Log in to save

As an AI Agentic Test Engineer at the core automation team, you will design and implement automated evaluation frameworks using Large Language Models to assess AI Agent quality and accuracy. Your day-to-day involves hands-on validation of agents in complex environments, automating web and API tests, and integrating these into CI/CD pipelines like Jenkins or Concourse. You must be an expert in Python for building custom test utilities and familiar with TypeScript, Java, and React for cross-functional tooling. This role requires 5+ years of experience as an SDET or Automation Engineer, a strong background in custom automation frameworks, and proficiency in AI agent evaluation techniques. The team focuses on delivering reliable, secure, and scalable agentic systems for Xfinity applications, aiming to transition from human-assisted workflows to fully AI-driven chat experiences.

Skills

Python CI/CD Jenkins Concourse React Streamlit TypeScript Java LLM-as-a-Judge Agent Evaluation Web Automation API Automation

What you'll do

Design and implement automated evaluation frameworks using Large Language Models to assess AI Agent quality.
Perform hands-on validation of AI Agents in complex environments for intelligent, semantic testing.
Develop Python scripts and automation tools for custom test utilities and framework extensions.
Build high-performance automated tests for modern web interfaces and RESTful APIs to ensure reliability.
Integrate automated tests into CI/CD pipelines to provide continuous quality feedback during development.

What we're looking for

5+ years of experience as an SDET or Automation Engineer.
Expert-level hands-on coding in Python for automation and testing.
Strong background in building and maintaining custom automation frameworks.
Proven experience in Web and API automation across modern tech stacks.
Solid understanding of AI Agent Evaluation techniques using LLMs as judges.
Ability to integrate automated tests into CI/CD pipelines like Jenkins or Concourse.

Similar roles

Agentic AI Engineer

Booz Allen Hamilton

Washington, DC +1 36 days ago $99,000–$225,000

LangChain AutoGen PydanticAI CrewAI LlamaIndex MCP A2A RAG Neo4j NebulaGraph Hugging Face PEFT LoRA Grafana Langfuse LangSmith Phoenix Docker Kubernetes AWS Azure ONNX GGML Ollama

Save

Agentic AI Evaluation Engineer

Comcast

Washington, District Of Columbia 1 day ago $142,651–$213,977

CI/CD Machine Learning Large Language Models Python SQL PostgreSQL Docker Kubernetes Terraform AWS Prometheus Grafana Git Jenkins

Save

Agentic AI Engineer

Booz Allen Hamilton

Washington, DC +2 51 days ago $99,000–$225,000

LangChain AutoGen LangGraph Model Context Protocol MCP RAG Knowledge Graphs Neo4j NebulaGraph Hugging Face PEFT LoRA ONNX GGML Ollama ReAct loops microservice design edge computing CI/CD

Save

Engineer 3, Agentic AI

Comcast

Philadelphia, PA +1 1 day ago

Python Agent Development Kit (ADK) Gemini Enterprise MLOps LLM-as-a-Judge Spec-driven development AI-assisted coding Prompt engineering AgentOps Production monitoring Google-centric ecosystem

Save

Distinguished AI Engineer (Agentic AI Platform)

Capital One Financial

San Jose, CA +4 35 days ago $269,100–$307,200

Python Go Kubernetes LangChain AutoGen SemanticKernel LLMOps GoogleCloudVertexAI AmazonSageMaker AzureMachineLearning CI/CD ResponsibleAI DataPrivacy MultiTenantSecurityPatterns Prometheus Grafana

Save

AI Agent Engineer

General Motors (GM)

Remote (Gm Global Technical Center - Michigan It Innovation Center) 42 days ago

Python TypeScript JavaScript Java LLMs RAG RESTful APIs LangChain Semantic Kernel Glean M365 RBAC Observability Telemetry A/B Testing CI/CD

Remote Hybrid

Save