Principal System Engineering

AT&T

Quick summary

Work type
On-site
Location
Atlanta, GADallas, TXPlano, TXMiddletown, NJ
Salary
$155,400–$261,100 / yr
Posted
today

Market check

Salary context

Competitive pay

How this pay compares to similar roles

Similar $193k
This role $208k
$129k most similar roles pay here $275k

This role pays more than 56% of similar roles. Most pay $164,350–$222,000 — the shaded band above. At the midpoint, this role pays about $208k versus about $193k for comparable roles.

Based on 239 similar postings.

Employer

About AT&T

AT&T is a US-based telecommunications company providing wireless, broadband, and fiber internet service along with phone and connectivity products for consumers and businesses.

AT&T currently has 78 open roles on FindRole.

Listed pay typically runs $128,400–$192,600 across 74 roles with salary data.

Most-posted roles

View all roles at AT&T

At a glance

TL;DR · Principal System Engineering

As a Principal Systems Engineer at AT&T’s Corporate Systems team, you will play a crucial role in enhancing system reliability and software delivery by analyzing production incidents across applications, infrastructure, and cloud environments. Your day-to-day responsibilities include conducting deep root cause analysis (RCA), identifying patterns, and driving preventive actions to minimize recurrence of issues. You will work closely with engineering teams to implement permanent fixes and long-term improvements, leveraging observability tools like logs, metrics, traces, and AI-assisted methods for data analysis. Essential skills include a strong understanding of end-to-end system architecture, experience in SRE tools such as T-APM, Grafana, and CI/CD processes, proficiency with Python, SQL, and ITSM tools like ServiceNow, and the ability to write clear postmortems. This role requires 7+ years of relevant experience in Systems Engineering or QA, and a background in SRE, Support, or Test Engineering is preferred.

What you'll do

  • Conduct root cause analysis for production incidents to prevent recurrence.
  • Analyze end-to-end system architecture using observability data and tools.
  • Write clear, structured postmortems after each incident for continuous improvement.
  • Partner with engineering teams to implement permanent fixes and preventive measures.
  • Use AI-assisted methods to analyze operational data and identify patterns.

What we're looking for

  • Proven experience performing deep RCA for production incidents.
  • Strong understanding of end-to-end system architecture including cloud, web apps, APIs, databases, and infrastructure.
  • Hands-on experience with observability tools such as logs, metrics, traces.
  • Ability to identify patterns and drive preventive actions in incident management.
  • Experience writing clear, structured postmortems and partnering with engineering teams for improvements.
  • Strong systems thinking and problem-solving skills combined with data analysis capabilities.
  • 7+ years of experience in Systems Engineering, ITSM, RM/CM, SRE, Support or QA.

More like this

Similar roles

Principal System Engineering

AT&T

Plano, TX +3 today $155,400$261,100
Grafana T-SQL Python PowerBI ServiceNow CI/CD DevOps Agile SAFe AI DataAnalytics JiraAlign Git Tableau ObservabilityTools Postmortems

Principal System Engineering

AT&T

Dallas, TX +3 today $155,400$261,100
Grafana T-SQL Python PowerBI ServiceNow CI/CD DevOps Agile SAFe AI DataAnalytics JiraAlign Git Tableau

Principal Systems Engineer

Motorola Solutions

Schaumburg, IL 102 days ago $130,100$170,125
GitHub Jira Confluence Android Linux RF FPGA RTOS Configuration Management Embedded OS systems Wireless communications technology Digital Hardware Multitasking Prioritization Verbal communication Written communication Interpersonal skills Requirements management Test management

Principal Engineers, Systems Design

T-Mobile

Herndon, VA 4 days ago $161,782$202,900
5G 3GPP RAN Network Architecture Device OEM RAN Features Security Testing Project Management Proof of Concepts Technical Documentation Cyprus Telecommunications Standards Vendor Roadmaps Advanced Engineering Techniques System Interactions Performance Reporting Optimization Techniques Executive Presentations
Hybrid

Principal Engineer

Intuit

Mountain View, CA 77 days ago $245,500$332,500
Python Java Kubernetes Docker AWS CI/CD PostgreSQL Redis MongoDB Git Jenkins SonarQube Selenium JUnit

Principal Engineer

Qualcomm

San Diego, CA 66 days ago $200,800$301,200
Python C++ TensorFlow PyTorch Linux Android Computer_Vision Deep_Learning Kubernetes Docker CI/CD Prometheus Grafana Qualcomm_Accelerators INT8 FP16 Heterogeneous_Compute AR/VR Robotics Autonomous_Vehicles IoT