Principal Security Research Manager

What you'll do

• Design and build end-to-end evaluation harnesses for agentic security and engineering workflows.
• Create representative benchmark suites and golden datasets including adversarial and production-derived cases.
• Implement deterministic validators, automated graders, and trace analyzers to make evaluations repeatable.
• Measure task success, safety compliance, latency, token usage, and total cost per outcome.
• Compare models, prompts, and orchestration patterns to identify quality-versus-efficiency tradeoffs.
• Integrate evaluations into CI/CD pipelines and release gates to provide evidence-based production decisions.
• Connect offline evaluation results with production feedback like human overrides and escaped defects.
• Convert evaluation learnings into reusable playbooks, templates, and dashboards to scale across the organization.

What we're looking for

• Must have a Doctorate with 3+ years, Master's with 4+ years, or Bachelor's with 6+ years of relevant experience in software development, cybersecurity, or related fields.
• Must have a Doctorate with 5+ years, Master's with 8+ years, or Bachelor's with 12+ years of relevant experience (preferred).
• Must have at least 1 year of people management experience.
• Must have equivalent experience to the educational and years-of-experience requirements.
• Proven software engineering experience building production systems, developer platforms, test infrastructure, automation frameworks, data pipelines, or reliability tooling (preferred).
• Ability to design and implement end-to-end evaluation systems including task definition, dataset creation, harness implementation, scoring, and operational integration (preferred).
• Demonstrated coding, debugging, system design, and operational excellence skills (preferred).
• Experience with LLMs, AI agents, model evaluation, prompt/tool orchestration, automated grading, or benchmark design (preferred).