Scope of the Role: We are looking for a Technical Training & Quality Manager - AI Data to lead technical training, capability development, quality assurance, and continuous improvement for AI data programs.
The role will work closely with AI/ML, Data Science, Operations, Program Management, and Quality teams to ensure that teams working on data annotation, data preparation, model evaluation, RLHF, LLM training, AI response evaluation, and other AI-data workflows have the required technical capabilities and consistently meet client-defined quality standards.
The ideal candidate should combine strong technical understanding of AI/ML and data workflows with hands-on experience in training, quality management, process improvement, and large-scale operations
What You'll Own:Technical Training & Capability Development:
- Design and execute technical training programs for AI data and annotation teams.
- Develop training curricula, learning paths, assessments, certification programs, and refresher modules.
- Train teams on AI/ML concepts, LLMs, Generative AI, NLP, data annotation, data labeling, model evaluation, prompt engineering, RLHF, and AI response quality.
- Conduct Train-the-Trainer programs and build internal technical trainers.
- Identify skill gaps through assessments, production performance, and quality metrics and create targeted upskilling plans.
- Develop practical exercises, technical assessments, simulations, and certification frameworks.
Quality Management:
- Own quality frameworks and standards across AI data projects.
- Define and monitor quality KPIs, accuracy, agreement rates, defect rates, audit scores, rework, and productivity.
- Establish quality calibration processes and conduct regular quality audits.
- Analyze quality trends and identify root causes of recurring defects.
- Partner with Operations and Program Managers to implement corrective and preventive actions.
- Drive continuous improvement initiatives to improve accuracy, consistency, productivity, and turnaround time.
AI Data & Technical Operations :
Provide technical guidance for projects involving:
- Data annotation and labeling
- LLM evaluation
- RLHF / human feedback
- Prompt-response evaluation
- NLP and text classification
- Image/video/audio annotation
- Generative AI evaluation
- Data validation and enrichment
- Model benchmarking and red teaming
- Understand project guidelines, client specifications, annotation taxonomies, and evaluation rubrics and translate them into effective training and quality programs.
- Work with SMEs and technical teams to resolve complex quality and interpretation issues.
Stakeholder Management:
- Work closely with clients, Program Managers, Operations, Engineering, Data Science, and QA teams.
- Participate in client calibration sessions and quality reviews.
- Present quality dashboards, training effectiveness, RCA findings, and improvement plans to senior leadership.
- Support new project launches through training needs analysis, SOP development, quality framework creation, and readiness assessments.
Continuous Improvement:
- Identify opportunities to improve training effectiveness, operational quality, and process efficiency.
- Use data and analytics to measure training ROI and quality improvement.
- Drive automation and technology adoption in training and quality processes.
- Standardize best practices across projects and delivery teams.
You'll Thrive in This Role If You Have:- 8-12 years of experience in AI/ML, data operations, data annotation, AI training, quality management, technical L&D, or related areas.
- Bachelor's/Master's degree in Computer Science, Engineering, Data Science, AI/ML, Statistics, or a related field.
- Strong understanding of Artificial Intelligence, Machine Learning, Generative AI and LLMs.
- Experience working with AI data / annotation / model evaluation projects.
- Experience managing training and quality teams in a high-volume delivery environment.
- Strong analytical and problem-solving skills.
- Experience with Root Cause Analysis, CAPA, calibration, quality audits and process improvement.
- Strong stakeholder and client management skills.
- Excellent communication, presentation, and facilitation skills.
- Ability to convert complex technical concepts into easy-to-understand training content.
- Certifications in AI/ML, Quality Management, Six Sigma, instructional design or technical training are preferred.
- Experience with AI platforms, annotation tools, LLM evaluation frameworks, or data-quality platforms.
- Exposure to Python, SQL, analytics/BI tools, or automation would be an advantage.
The expected salary range for this position is $145,000 - $175,000 p/year, based on experience, skills, and qualifications.
