Principal Product Manager

Microsoft

RemotePosted July 29, 2026

About the role

Overview: Copilot's usefulness depends on reliably meeting user needs safely across a wide range of enterprise and consumer scenarios. We are seeking a Principal Product Manager to own how we measure that. You will build and run evaluation systems to determine whether a model update, a new Copilot skill, or an agentic workflow is ready to ship. You will partner with applied research, model training, and engineering teams to translate that question into concrete, defensible metrics and use those metrics to shape what we prioritize next. This is a technical, judgment-heavy PM role. You will not only write requirements but also analyze transcripts, build evaluation sets, reason about what good looks like for a given surface, and decide whether something is ready for customers.

Principal Product Manager

Microsoft

RemotePosted July 29, 2026

About the role

Overview: Copilot's usefulness depends on reliably meeting user needs safely across a wide range of enterprise and consumer scenarios. We are seeking a Principal Product Manager to own how we measure that. You will build and run evaluation systems to determine whether a model update, a new Copilot skill, or an agentic workflow is ready to ship. You will partner with applied research, model training, and engineering teams to translate that question into concrete, defensible metrics and use those metrics to shape what we prioritize next. This is a technical, judgment-heavy PM role. You will not only write requirements but also analyze transcripts, build evaluation sets, reason about what good looks like for a given surface, and decide whether something is ready for customers.

Responsibilities

Own the evaluation lifecycle end-to-end: design, build, and maintain evaluation suites (offline benchmarks, live A/B experiments, agentic task suites, red-team/adversarial sets) that measure quality, safety, and task success across Copilot surfaces (Microsoft 365, Windows, Edge, Copilot Studio agents, etc.).
Gate model and feature launches: define entry and exit criteria for shipping model updates or new capabilities, and make the ship/no-ship call in partnership with research and engineering leadership.
Translate model behavior into product requirements: read outputs and failure modes directly, identify patterns, and turn them into prioritized fixes whether that is a training data gap, a prompt/system-message change, or a UX guardrail.
Build feedback loops: stand up pipelines

Track similar jobs

Get email alerts when new roles like this are posted.

Based on: Principal Product Manager

See how this role matches your resume

Sign in to get an AI match score and personalized feed.

Create account Sign in

More in Remote-first

Product Engineering Fintech Remote Europe Startups