We are looking for a Technical Project Manager with extensive experience working with external vendors & projects focused on data labeling and data annotation.
Essential functions
- End-to-End Project Lifecycle: Own the planning, scheduling, execution, and delivery of high-volume data annotation projects. Translate model training data needs into structured project timelines and milestones.
- Agile Operational Workflows: Implement project management methodologies (e.g., Agile, Scrum, Kanban) to track progress, unblock pipeline bottlenecks, and manage backlogs.
- Resource & Capacity Planning: Forecast resource needs, track operational budgets, and align data deliveries with the release cycles of the Data Science and ML Engineering teams.
- Risk Mitigation: Proactively identify project risks (such as edge cases, delivery delays, or drop-offs in accuracy) and establish contingency plans.
- Vendor Relations: Serve as the primary point of contact for external Business Process Outsourcing (BPO) partners, labeling vendors, and global crowd-sourced workforces.
- SLA & Performance Tracking: Define, monitor, and enforce service-level agreements (SLAs) and key performance indicators (KPIs) regarding vendor throughput, speed, and cost efficiency.
- Onboarding & Scaling: Lead the selection, negotiation, and onboarding processes for new data vendors to strategically scale operational capacity as company needs evolve.
- Guideline Architecture: Collaborate with engineering teams to build and refine airtight, unambiguous annotation guidelines, taxonomies, and training materials.
- Quality Assurance Frameworks: Design and enforce rigorous data auditing processes, measuring metrics like Inter-Annotator Agreement (IAA), consensus scores, and precision/recall of human labels.
- Tooling Optimization: Evaluate and manage data labeling tools and platforms (e.g., Labelbox, Scale AI, CVAT, or custom internal interfaces) to maximize annotator efficiency.
Qualifications
- Experience: 4+ years of professional experience in Data Operations, Technical Project Management, or MLOps, with a heavy emphasis on handling AI/ML training data.
- Project Management Skills: Demonstrated track record of managing complex, cross-functional projects from inception to completion.
- Data Annotation Background: Hands-on experience designing and executing text, image, video, audio, or multimodal data labeling workflows.
- Vendor Management: Experience directly managing, communicating with, and auditing third-party vendors or offshore operations.
- Analytical Capabilities: Strong data tracking habits with the ability to build performance dashboards and parse metrics via Excel/Google Sheets or SQL.
- Bachelor's degree in Computer Science, a related technical field, or equivalent practical experience.
Would be a plus
- Experience with advanced labeling workflows, such as RLHF (Reinforcement Learning from Human Feedback), red-teaming, or prompt-response evaluation for Large Language Models (LLMs).
- Basic understanding of Python or automation scripts to seamlessly move data across different systems and APIs.
- Experience working in a fast-paced, high-growth environment where workflows iterate rapidly.
We offer
- Opportunity to work on cutting-edge projects
- Work with a highly motivated and dedicated team
- Competitive salary
- Flexible schedule
- Benefits package - medical insurance, vision, dental, etc.
- Corporate social events
- Professional development opportunities
- Well-equipped office
