The Product Data Operations (PDO) team provides the data, insights, and human operational rigor that power machine learning and AI at Meta. Within the Family of Apps (FoA), we partner with Product, Engineering, Research, and Data Science to deliver high-quality data and evaluations that support product development and improve experiences across Meta's apps. As a Program Manager on the Product Data Operations Family of Apps team, you will independently lead multiple cross-functional programs that deliver high-quality data and evaluate AI model performance through AI-powered production and specialized human expertise. You will translate product needs into executable plans, determine the right mix of AI and expert-human workflows, and own program quality and delivery outcomes. In partnership with technical teams and domain experts, you will benchmark AI labelers, establish and maintain trusted evaluation datasets, calibrate automated judges, and monitor performance as models and product needs evolve. You will strengthen expert-reviewer capabilities and improve the tools, measurement practices, and workflows that enable reliable, scalable delivery. Throughout, you will turn operational and evaluation findings into actionable recommendations for product partners while improving quality, speed, and cost. This individual-contributor role requires sound judgment in ambiguous situations, analytical problem solving, and the ability to influence internal partners and external delivery teams without formal authority.
Responsibilities
Own multiple data labeling, expert human review, and/or AI evaluation programs end-to-end, including requirements, milestones, dependencies, risks, and delivery plans
• Partner with Product, Engineering, Research, and Data Science to translate product objectives into dataset requirements, evaluation criteria, and measurable program outcomes
• Define and track success metrics for data quality, throughput, turnaround time, and operational efficiency; use results to identify gaps and drive improvements
• Manage vendor and contractor delivery workflows and performance, including task instructions, calibration, quality reviews, and issue resolution
• Design and improve human-driven evaluation processes to assess model outputs, identify recurring failure patterns, and provide actionable feedback to product and engineering partners
• Identify, pilot, and scale LLM-assisted labeling and AI-enabled operational workflows, measuring efficiency gains and validating quality through human review and clear escalation criteria
• Develop reusable intake templates, tracking mechanisms, and process playbooks that improve the consistency and scalability of program delivery
• Identify tooling and measurement gaps, define requirements with cross-functional partners, and drive adoption of solutions within owned programs
• Analyze operational data and evaluation findings to recommend improvements to products, datasets, and workflows
• Align stakeholders on priorities and trade-offs, communicate progress and risks, and resolve cross-functional blockers to keep programs on track
Minimum Qualifications
• Bachelor's degree in a directly related field, or equivalent practical experience
• 6+ years of experience in program or project management, operations, consulting, or a related role in a technology or consumer-product environment
• Experience independently leading multiple cross-functional programs from scoping through delivery, including managing dependencies, risks, and competing priorities
• Experience defining success metrics, analyzing data, and translating findings into decisions or measurable operational improvements
• Experience building relationships and influencing stakeholders across functional boundaries without formal authority
• Experience communicating program plans, results, risks, and recommendations to both operational teams and leadership
• Experience developing or improving repeatable processes and applying judgment in ambiguous, fast-paced environments
• Experience using Excel or Google Sheets for analysis, tracking, and reporting
Preferred Qualifications
• Understanding of Facebook, Instagram, and Threads products
• Experience integrating AI tools into operational workflows and demonstrating measurable improvements in efficiency or quality while applying responsible AI practices
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
• Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
• Experience partnering with Product and Engineering teams on consumer-facing products; familiarity with Facebook, Instagram, or other Meta Family of Apps products
• Experience managing data labeling, annotation, machine learning datasets, or human evaluation programs, including vendor or contractor delivery
• Experience with LLM evaluation techniques, including evaluation rubrics, reviewer calibration, error analysis, and human-in-the-loop quality assurance
• Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
• Experience with SQL and building or specifying dashboards, reporting tools, or performance measurement frameworks
