About the Team and the Role:The Senior Product Manager will be the connective tissue between category managers, merchandisers, and domain guides on one side, and the scientists and engineers building AI models on the other. This role requires a deep understanding of how AI models are built and trained - including how data curation, human judgment, and iterative feedback shape model behavior - and the ability to actively contribute to that process. Your impact depends on execution, trust, and the ability to coordinate complex cross-functional work across a matrix organization.
What you will accomplish:Run Human Evaluation and Annotation Programs
- Design and lead structured programs to evaluate AI model outputs across ranking, classification, generation, and extraction tasks
- Coordinate distributed annotation teams across timezones - running workflows, maintaining quality consistency, and ensuring continuous coverage and on-time delivery. Handle relationships with internal and external evaluator pools
- Translate evaluation results into clear, actionable requirements for model and product teams
Lead AI Quality Monitoring Programs
- Track quality metrics, surface issues proactively, and run structured resolution processes to ensure AI-driven experiences consistently meet the bar
- Ensure quality signals from the live site feed back into model development priorities.
- Review what ships, ask hard questions, and hold teams accountable when AI-driven experiences fall short
- Help define what "good data" looks like for each focus category: coverage, freshness, labeling schema, and quality thresholds
- Build the end-to-end processes to meet those standards - from scoping and sourcing through annotation, QA, and delivery - in partnership with science teams and business partners
- Define quality standards for training and evaluation data in partnership with business partners and science teams - and build the processes to meet those standards consistently.
What you will bring:- 5+ years of PM experience with deep exposure to AI/ML or technical platform products
- Experience leading annotation or evaluation programs, including working with distributed annotator pools, external vendors, or crowdsourced labeling at scale
- Analytical and data science fluency - comfortable writing SQL queries to explore datasets, track program metrics, and validate quality signals independently
- Ability to work alongside Applied Scientists - understands how DS workflows operate, can scope analytical work clearly, and knows when to roll up their sleeves.
- Working knowledge of how AI models are built and trained - including the role of data, human judgment, and evaluation in shaping model behavior
- Direct exposure to LLM evaluation - preference data, benchmarking, red-teaming, or human feedback collection is a plus
- Experience with annotation vendors or crowdsourced labeling programs is a plus
- Familiarity with AI observability, quality monitoring, or anomaly detection in production is a plus
Additional DetailsThe base pay range for this position is expected in the range below:
C$133,200 - C$177,800
Base pay offered may vary depending on multiple individualized factors, including location, skills, and experience. The total compensation package for this position may also include other elements, including a target bonus and restricted stock units (as applicable) in addition to a full range of medical, financial, and/or other benefits (including RRSP eligibility, various paid time off benefits, such as PTO and parental leave). Details of participation in these benefit plans will be provided if an employee receives an offer of employment.
This job posting relates to an existing vacancy within eBay.
