As a Senior AI/ML DevOps Engineer at Cisco’s CX AI Incubation Team, you will play a pivotal role in productionizing large language and semantic models for intelligent customer experiences across cloud and on-prem environments. Your day-to-day responsibilities include building robust deployment pipelines with clear SLAs, optimizing inference performance across various hardware configurations, packaging on-prem inference stacks securely, designing scalable serving architectures, and implementing model observability features. You will work closely with product and engineering teams to ensure reliable and secure AI services, leveraging technologies such as PyTorch/TensorFlow, Kubernetes, CI/CD pipelines, and GPU profiling tools like vLLM and TensorRT-LLM. This role demands expertise in Python, Java or C++, hands-on experience with NLP systems, and a track record of operationalizing models at scale, particularly in resource-constrained environments.
Build and deploy robust model-serving pipelines for LLM/SLM features with clear SLAs.
Optimize inference performance across various hardware configurations for cost efficiency.
Package on-prem inference stacks securely and integrate them into customer environments.
Design scalable serving architectures for multi-tenant, secure generative AI systems.
Implement automated CI/CD processes for models and prompts to ensure reproducibility.
Develop model and service observability features including latency metrics and safety checks.
Support training and fine-tuning workflows for LLMs/SLMs with data curation and tracking.
What we're looking for
7+ years of experience in AI/ML DevOps or related field.
Strong background in Python, Java, and C++ for building production services.
Experience with PyTorch/TensorFlow and ML lifecycle tooling.
Proven track record deploying and operating NLP/Generative AI systems.
Hands-on expertise in GPU-backed inference and runtime optimization.
Familiarity with Kubernetes, CI/CD pipelines, model registries, and observability tools.
Ability to work effectively in cross-functional teams and communicate technical concepts.
Market check
Salary context
This
$199,700–$254,600
range sits above
61%
of similar postings on FindRole.
Peer median band
$169,375–$261,300
Median floor and ceiling across peers.
Typical midpoint (25–75%)
$176,000–$246,150
Middle half of comparable postings.
Based on 240 comparable postings.*
* 240 is the maximum number of comparable postings sampled.
Employer
About Cisco
Cisco Systems is the world''s leading networking technology company, designing and manufacturing networking hardware, telecommunications equipment, and cybersecurity solutions for businesses and governments. Industry: Networking Technology & Cybersecurity
Cisco currently has
113 open roles
on FindRole.
Listed pay typically runs
$165,000–$241,400
across 113 roles with salary data.