3+ years of experience in AI/ML, data operations, or related fields.
Proven track record as a Technical Program Manager or strategic program lead at a startup.
Skilled in translating ambiguous technical requirements into actionable plans.
Expertise in assessing dataset quality through metrics and examples.
Experience managing external vendors and scaling delivery pipelines.
Familiarity with reinforcement learning and human-in-the-loop data processes.
Strong communication skills and ability to work independently in fast-paced settings.
Responsibilities
Lead data and evaluation programs from inception to delivery.
Translate ambiguous requests into clear specifications with defined milestones.
Enhance data quality processes through quantitative and qualitative measures.
Manage external vendors throughout the delivery lifecycle.
Streamline workflows, documentation, and automation for data production.
Collaborate with teams to integrate feedback into product improvements.
Benefits
Medical, dental, and vision coverage.
Retirement plan options.
Paid time off for work-life balance.
Commuter benefits to ease travel costs.
Visa sponsorship and relocation support for qualified candidates.
Full Job Description
About the Role
Own complex data and evaluation programs for AI research teams and internal partners. This role brings structure to evolving technical requirements and helps ensure that data and evaluation workflows produce reliable, useful results. What You'll Do
Lead data and evaluation programs from scoping and requirements gathering through production, quality assurance, delivery, and retrospectives.
Turn ambiguous requests into clear specifications with milestones, owners, dependencies, and acceptance criteria.
Improve data quality processes through quantitative measures and qualitative review, including assessment of task coverage, difficulty, and reward distributions.
Manage external vendors across the delivery lifecycle and improve delivery pipelines at scale.
Improve workflows, documentation, automation, and cross-functional handoffs for data production and evaluation.
Work with research, platform, and marketplace teams to turn feedback and learnings into product improvements.
What We're Looking For
At least 3 years of experience owning complex technical programs in AI/ML, data operations, support engineering, applied research, forward-deployed engineering, or a related field.
Experience as a Technical Program Manager or strategic program lead at a startup, with a focus on project delivery rather than product management alone.
Experience translating ambiguous technical requirements into executable plans and improving complex data quality processes.
Ability to assess dataset quality using both aggregate metrics and individual examples.
Experience managing external vendors, scaling delivery pipelines, and supporting AI research teams or technical enterprise customers with evolving needs.
Familiarity with RLHF, reinforcement-learning environments, agent evaluations, human-in-the-loop data pipelines, annotation, or expert data collection.
Strong written and verbal communication skills, sound judgment, and comfort working independently in a fast-paced environment.
Compensation & Benefits
Annual salary range: $130,000 to $225,000. Visa sponsorship and relocation support are available for qualified full-time candidates. Benefits include medical, dental, and vision coverage, a retirement plan, paid time off, and commuter benefits. Location
On-site in San Francisco, California, United States.
