Senior Software Engineer

What you'll do

• Develop automated evaluation coverage for new Siri features and interaction modalities from ambiguous requirements.
• Translate product goals into measurable system behaviors by building eval harnesses and test datasets based on real user workflows.
• Build and improve reward models and alignment signals to measure if responses meet user needs.
• Design and ship end-to-end evaluation tooling, pipelines, and architecture including data ingestion, scoring, and production monitoring.
• Diagnose failures across the stack from environment provisioning to pipeline execution to implement durable fixes.
• Implement "evals first" engineering by designing for observability, logging, and reproducibility in all development tools.

What we're looking for

• Strong programming skills in one or more compiled languages such as Swift, C++, or Objective-C.
• Strong Python skills and solid computer science fundamentals including data structures and algorithms.
• Experience with backend/API development and production debugging.
• Ability to learn and adapt to evolving technologies like GenAI tools and ML frameworks.
• Excellent communication and cross-team collaboration skills within large, cross-functional organizations.
• B.S. or M.S. in Computer Science, Machine Learning, or a related field (or equivalent experience).
• Experience evaluating ML, LLM, or agent-based systems including metrics, scoring methodology, and RAG techniques (preferred).
• Understanding of reinforcement learning, transformer architectures, and frameworks like PyTorch or Hugging Face (preferred).