5+ years in AI/ML data operations or production, with 2+ years managing project managers or team leads in a distributed setting.
Strong understanding of LLM training processes and evaluation methodologies.
Experience managing teams against KPIs for quality and cost-per-task, with proficiency in SQL for data analysis.
Proven track record of on-time delivery across multiple concurrent data collection programs.
Experience managing contractor workforce operations across various regions and languages.
Demonstrated use of Agile, Scrum, or Kanban methodologies for workflow management.
Strong communication skills in writing guidelines and reporting to leadership.
Responsibilities
Ensure every program has designated Project Managers and deliverables meet quality and timeline targets.
Source, onboard, and develop new Project Managers to meet demand and reduce attrition.
Implement quality interventions when issues arise during programs, utilizing QA loops and retraining.
Standardize processes and improve software by creating shared templates and implementing documented enhancements from post-mortems.
Identify and escalate risks proactively, ensuring timely resolution among cross-functional teams.
Benefits
Comprehensive health and wellness programs.
Flexible work environment with options for remote and hybrid work.
Opportunities for professional development and continued education.
Access to cutting-edge technology and tools for project management.
Full Job Description
Key Responsibilities
PM Performance
Outcome: Every active program has accountable Project Manager(s), and every PM carries a workload within the agreed span. Programs hit on-time delivery and first-pass acceptance targets without escalation; each PM is reviewed monthly against a scorecard of throughput, quality, and cost-per-task.
PM Hiring, Onboarding, and Development
Outcome: PM pool capacity keeps pace with signed demand: new PMs are sourced, onboarded, and running their first program within the agreed ramp window, all programs start on time, and PM attrition is below threshold.
Quality Interventions Across Programs
Outcome: Quality dips are caught mid-program through QA loops and corrected via retraining of annotator pools or guideline updates. Repeated misses by a PM or annotator pool lead to documented remediation or replacement. Issues are proactively discovered.
Process Standardization & Software Improvements
Outcome: Programs launch from shared playbooks, guideline templates, and dashboard standards rather than being rebuilt per engagement. Every post-mortem produces documented improvements that lead directly into our custom software stack, and time from program handoff to first delivery declines quarter over quarter.
Escalation and Cross-Functional Interface
Outcome: Risks surface to Technical Program Managers early enough to be managed. Escalations between the PM pool, Quality, Talent, and Delivery are resolved within agreed timelines.
Qualifications
People management in AI data operations: 5+ years in AI/ML data operations or production, including 2+ years directly managing project managers or team leads in a distributed, multi-time-zone contractor environment.
LLM knowledge: Strong understanding of LLM training processes (pre-training, SFT, RLHF) and evaluation methodologies (human-in-the-loop, red teaming), and of what drives quality and throughput in annotation workflows.
KPI-driven management: Has run teams against throughput, quality (accuracy, IAA, gold-set), and cost-per-task targets; advanced proficiency with spreadsheets and dashboards, and able to use SQL to extract and analyze performance data.
Delivery track record: Has sustained on-time delivery and acceptance targets across multiple concurrent data collection or evaluation programs for enterprise or research lab customers.
Contractor workforce operations: Has hired, ramped, performance-managed, and offboarded hourly and freelance staff across regions and languages.
Methodology: Proven track record using Agile, Scrum, or Kanban to manage complex workflows across a portfolio of programs.
Communication: Writes clear, unambiguous guidelines and feedback for multilingual audiences and communicates status, risk, and tradeoffs crisply to leadership.
Preferred Skills
Fluency in multiple human languages.
Experience with multilingual data deliveries (pre-training, SFT, RLHF, machine translation, multimodal, etc.), especially in rare-resource languages
Experience with data annotation platforms (e.g., Label Studio, SuperAnnotate) and project management tooling (e.g., Jira).
Background in ML engineering, computer science, or data science.
