5+ years in AI/ML data operations, including 2+ years managing project managers in a multi-time-zone environment.
Strong knowledge of LLM training processes and evaluation methodologies.
Experience managing teams against throughput, quality, and cost-per-task metrics; proficient in data analysis using SQL.
Demonstrated history of maintaining on-time delivery in concurrent programs for enterprise or research clients.
Experience in hiring and managing remote contractors for diverse projects across languages.
Proven ability to utilize Agile, Scrum, or Kanban methodologies for complex project management.
Strong communication skills for crafting guidelines and updating stakeholders on project status.
Responsibilities
Ensure each program has an accountable Project Manager and that all programs meet on-time delivery and quality targets.
Source and onboard new Project Managers, enabling them to run programs independently within defined timelines.
Identify quality issues during programs and implement corrective measures through retraining or guideline updates.
Standardize processes using shared templates and playbooks while integrating improvements into the software stack after every project.
Manage escalations and risks by facilitating early communication between the Project Manager pool, Quality, Talent, and Delivery teams.
Benefits
Opportunities for professional development and continuous learning.
Access to a collaborative and dynamic work environment.
Flexible working hours to accommodate remote work across different time zones.
Involvement in cutting-edge AI and ML projects.
Potential for leadership roles in future initiatives.
Full Job Description
Key Responsibilities
PM Performance
Outcome: Every active program has accountable Project Manager(s), and every PM carries a workload within the agreed span. Programs hit on-time delivery and first-pass acceptance targets without escalation; each PM is reviewed monthly against a scorecard of throughput, quality, and cost-per-task.
PM Hiring, Onboarding, and Development
Outcome: PM pool capacity keeps pace with signed demand: new PMs are sourced, onboarded, and running their first program within the agreed ramp window, all programs start on time, and PM attrition is below threshold.
Quality Interventions Across Programs
Outcome: Quality dips are caught mid-program through QA loops and corrected via retraining of annotator pools or guideline updates. Repeated misses by a PM or annotator pool lead to documented remediation or replacement. Issues are proactively discovered.
Process Standardization & Software Improvements
Outcome: Programs launch from shared playbooks, guideline templates, and dashboard standards rather than being rebuilt per engagement. Every post-mortem produces documented improvements that lead directly into our custom software stack, and time from program handoff to first delivery declines quarter over quarter.
Escalation and Cross-Functional Interface
Outcome: Risks surface to Technical Program Managers early enough to be managed. Escalations between the PM pool, Quality, Talent, and Delivery are resolved within agreed timelines.
Qualifications
People management in AI data operations: 5+ years in AI/ML data operations or production, including 2+ years directly managing project managers or team leads in a distributed, multi-time-zone contractor environment.
LLM knowledge: Strong understanding of LLM training processes (pre-training, SFT, RLHF) and evaluation methodologies (human-in-the-loop, red teaming), and of what drives quality and throughput in annotation workflows.
KPI-driven management: Has run teams against throughput, quality (accuracy, IAA, gold-set), and cost-per-task targets; advanced proficiency with spreadsheets and dashboards, and able to use SQL to extract and analyze performance data.
Delivery track record: Has sustained on-time delivery and acceptance targets across multiple concurrent data collection or evaluation programs for enterprise or research lab customers.
Contractor workforce operations: Has hired, ramped, performance-managed, and offboarded hourly and freelance staff across regions and languages.
Methodology: Proven track record using Agile, Scrum, or Kanban to manage complex workflows across a portfolio of programs.
Communication: Writes clear, unambiguous guidelines and feedback for multilingual audiences and communicates status, risk, and tradeoffs crisply to leadership.
Preferred Skills
Fluency in multiple human languages.
Experience with multilingual data deliveries (pre-training, SFT, RLHF, machine translation, multimodal, etc.), especially in rare-resource languages
Experience with data annotation platforms (e.g., Label Studio, SuperAnnotate) and project management tooling (e.g., Jira).
Background in ML engineering, computer science, or data science.
