Software Engineer, Evaluation Engineering

What you'll do

• Architect and build high-reliability execution harnesses for multi-step agentic trajectories on target Apple hardware.
• Implement event-capture layers to record high-fidelity execution traces including tool inputs, outputs, and state mutations.
• Develop CLI tools and automation scripts to integrate on-device harnesses into local development and testing pipelines.
• Solve systems engineering challenges regarding local device state management, process isolation, and determinism.
• Optimize harness performance to ensure minimal impact on device thermal limits and memory profiles.
• Design infrastructure for high-fidelity trace generation across diverse hardware like iPhone, Mac, and Vision Pro.
• Build environment control layers to manage state resets and sandboxed test environments.

What we're looking for

• Proficiency in compiled languages such as Swift, C++, or Objective-C.
• Experience with Python for scripting and automation tasks.
• A B.S. or M.S. in Computer Science, Machine Learning, or equivalent experience.
• Solid knowledge of computer science and systems fundamentals, including process management and local runtimes.
• Experience developing on Apple platforms or working with low-level OS runtimes and frameworks.
• Experience designing environment-staging systems like state reset mechanisms or sandboxed test environments.
• Experience building telemetry, trace-capture, or event-logging infrastructure for multi-threaded or asynchronous systems.
• Familiarity with on-device AI/LLM runtime environments and agentic system architectures.