AI Product Management, Copilot

What you'll do

• Develop and execute on Copilot's LLM platform strategy to enhance language model capabilities.
• Prototype approaches to improve response quality across various scenarios by steering language models.
• Identify and prioritize issues affecting quality, factuality, and safety of language models; work with teams for resolution.
• Define and build evaluations using relevant datasets to demonstrate improvements in language model performance.
• Deploy experiments in production to drive measurable enhancements in relevance and engagement for Copilot users.
• Partner with product teams to scale tool building and resolve dependencies with inference, agents, and orchestration teams.

What we're looking for

• 5+ years of product management experience or equivalent.
• 3+ years leading ambiguous areas, defining requirements, and executing roadmaps.
• 2+ years building ML-powered or LLM-powered products.
• Experience with LLM APIs, embeddings, vector databases, and tool use.
• Hands-on expertise in prompt design, context window management, and model evaluation.
• Ability to work effectively with multi-disciplinary teams.