Data Annotation Lead

Physical Intelligence

$120K — $160K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • 7+ years leading large-scale data or annotation operations with teams of 100+
  • 3+ years experience managing managers
  • Proven track record of launching annotation programs from scratch
  • Expertise in annotation best practices and operational strategy
  • Experience integrating autolabeling and model-based workflows
  • Fluency with operational metrics for data-driven management
  • Strong collaboration skills with product and engineering teams
  • Effective communication skills to maintain standards across large teams

Responsibilities

  • Oversee all aspects of annotation operations: throughput, quality, cost, and delivery
  • Scale workforce from hundreds to thousands, focusing on workforce planning and hiring
  • Build and lead a multi-layer management team with a focus on development
  • Enhance throughput through autolabeling and model-guided annotation
  • Establish training and certification programs for new annotators
  • Define and improve quality standards through audits and calibrations
  • Create operational metrics and reports to drive continuous improvement
  • Manage capacity planning against competing demands
  • Implement performance management and feedback systems
  • Collaborate with product and engineering for optimal annotation tools
  • Translate annotation needs into clear instructions and SLAs
  • Manage budget and costs for the annotation operations

Benefits

  • Opportunity to lead and scale a pivotal operation
  • Collaborative environment with cross-functional partnerships
  • Focus on integrating innovative technologies in annotation
  • Potential to define best practices in the field
  • Impact on improving model performance through quality annotation
Full Job Description
The role

We're looking for a Data Annotation Lead to own annotation operations and scale the team behind it. Annotation is core to how our models improve, and demand is growing fast. You will scale the annotation workforce from 100s to 1,000s while raising the quality bar - designing the org, the training pipeline, the quality system, and the metrics that let it scale efficiently.

You will own the people and the operation: throughput, quality, cost, and delivery across every annotation type.

In this role you will
  • Own annotation operations end-to-end: throughput, quality, cost, and on-time delivery across all annotation types.
  • Scale the annotation workforce from 100s to 1,000s: workforce planning, org design, and the hiring and onboarding funnel.
  • Build and lead a multi-layer management structure; hire, develop, and manage managers and team leads.
  • Scale throughput with autolabeling and model-based annotation: design human-in-the-loop workflows where models pre-label and annotators review, correct, and escalate, so output grows faster than headcount.
  • Stand up the training and certification pipeline that brings new annotators and teams to the quality bar quickly and consistently.
  • Define and continuously raise the quality bar: rubrics, calibration, audit/QA loops, and quality-adjusted productivity.
  • Establish operational metrics and reporting (presence, throughput, acceptance/rejection, rework) and drive week-over-week improvement.
  • Run capacity planning and prioritization against competing demand; allocate teams to the highest-impact work.
  • Manage performance at scale with clear standards, feedback, and a fair improvement/exit process.
  • Partner with product and engineering to define annotation tooling that unlocks throughput and quality.
  • Partner with research and project leads to translate annotation needs into clear instructions, rubrics, and SLAs.
  • Own the in-house vs. vendor mix and manage external partners where used.
  • Own the annotation operating budget and unit economics; improve cost-per-annotation while protecting quality.

What you'll bring
  • 7+ years leading scaled data or annotation operations, including teams in the 100s+.
  • 3+ years as a manager of managers.
  • Track record standing up 01 annotation programs.
  • Deep command of annotation best practices, operations, and strategy.
  • Experience integrating autolabeling and model-based annotation into human workflows; building human-in-the-loop pipelines that raise throughput without sacrificing quality.
  • Fluency with operational and quality metrics; data-driven management of large workforces.
  • Strong cross-functional partnership with product, engineering, and research/ML.
  • Clear written and verbal communication; able to set and hold standards across a large, distributed team.
  • Working understanding of ML and why annotation quality drives model performance.

Nice to have
  • Experience in robotics, autonomous vehicles, or frontier-AI data pipelines.
  • Experience managing distributed/global and/or vendor workforces.
  • Built annotation tooling or partnered tightly with a tooling team.
  • Experience training or fine-tuning autolabeling models, or partnering closely with the ML teams that do.


Similar Jobs

More Jobs at Physical Intelligence

  • Data Annotation Lead
    $120K — $160K *
    San Francisco, CA 94112 (San Francisco County)
    Information Technology
    In-Person
  • Strategic Product Lead
    $120K — $150K *
    San Francisco, CA 94112 (San Francisco County)
    Enterprise Technology
    In-Person
  • Product Manager
    $120K — $150K *
    San Francisco, CA 94112 (San Francisco County)
    Enterprise Technology
    In-Person
  • Applied Researcher
    $120K — $150K *
    San Francisco, CA 94112 (San Francisco County)
    Consumer Technology
    In-Person
  • AI and Robotics Recruiter
    $100K — $150K *
    San Francisco, CA 94112 (San Francisco County)
    Technical Services
    In-Person

More Information Technology Jobs

Find similar Data Annotation Lead jobs: