Bachelor's degree in a quantitative or business field (e.g., business, finance, economics, statistics, mathematics, project management)
2 to 4 years of experience in business analytics, project management, or similar roles
Strong analytical skills to interpret conversation data and identify trends
Working knowledge or strong interest in AI/chatbot technologies and natural language processing
Ability to translate data findings into actionable recommendations
Project management skills to manage cross-functional inputs and track initiatives
Strong written and verbal communication skills for reporting and presentations
Attention to detail, especially in QA/testing for AI workflows
Responsibilities
Lead chatbot performance optimization by analyzing transcripts and interaction data
Report on continuous improvement actions and their impact during system evaluations
Design testing and QA for AI workflow launches, including test plans and acceptance criteria
Identify performance gaps and recommend improvement initiatives to leadership
Execute AI performance improvement initiatives, coordinating with cross-functional partners
Establish and maintain AI performance measurement frameworks and reporting standards
Analyze AI performance trends and communicate findings to leadership
Serve as a subject-matter expert on chatbot performance for support teams
Benefits
Comprehensive benefits package
401k contribution
Participation in a short-term incentive plan (subject to terms and policies)
Full Job Description
Essential Duties and Responsibilities:
Lead the chatbot performance optimization workstream: analyze conversation transcripts and quantitative interaction data to diagnose root causes of performance issues, then design and implement retraining and response improvements
Report on the actions taken and resulting impact of continuous improvement implementations during system performance evaluations
Design testing and QA for AI workflow launches and updates across support channels, including test plans, test data sets, and acceptance criteria, and recommend release readiness
Identify and document performance gaps, customer experience friction points, and automation opportunities, recommend improvement initiatives and business cases to leadership
Lead execution of AI performance improvement initiatives, coordinating cross-functional partners and adjusting scope or approach when performance results indicate an initiative is not delivering the intended impact • Partner with Knowledge Management to define standards for AI- ready knowledge sources (accuracy, currency, structure and drive remediation where content limits chatbot performance
Establish and maintain the AI performance measurement framework, including metric definitions, reporting standards, and measurement methods
Analyze and communicate the drivers behind changes, anomalies, trends, and results in AI performance, report the actions taken and their measured impact, with recommendations to leadership
Serve as the subject-matter resources on chatbot performance for Customer Experience and Technical Support teams, advising on conversation design, quality standards, and best practices for AI-assisted support
Comply with health and safety guidelines and rules; managers should also ensure compliance across their teams.
Protect Chamberlain Group's reputation by keeping information confidential.
Maintain professional and technical knowledge by attending educational workshops, reading professional publications, establishing personal networks, and participating in professional societies
Contribute to the team effort by accomplishing related results and participating on projects as needed
Minimum Qualifications
Bachelor's degree in a quantitative or business field (business, finance, economics, statistics, mathematics, project management)
2 to 4 years' work experience in business analytics, project management, or similar functions
Strong analytical skills, with the ability to interpret conversation data and quantitative results to identify trends, anomalies, and root causes
Working knowledge of or strong interest in AI/chatbot technologies and natural language processing concepts
Ability to translate data findings into clear, actionable recommendations for chatbot retraining and system calibration
Project management skills, including the ability to manage cross-functional inputs and track improvement initiatives to completion
Strong written and verbal communication skills, with the ability to prepare reports and present performance results to stakeholders and leadership • Comfort working with reporting tools and data organized from multiple systems
Ability to collaborate cross-functionally (e.g., with Knowledge Management, QA, and leadership) to align on priorities and standards
Strong attention to detail, particularly in QA/testing contexts for AI workflow launches and updates
Adaptability and comfort operating in a fast-evolving, technology-driven environment • Proficiency in Microsoft Office applications
Preferred Qualifications: Project Management, Experience working in a large, matrixed team, Effective partner management, negotiation, and operational skills with strong results orientation and efficiency mentality, Analytical skills, Project management skills, In-depth knowledge of Chamberlain Group products, policies, and procedures
#LI-JE1
The pay range for this position is $78,597.00 - $129,534.50; base pay offered may vary depending on a number of factors including, but not limited to, the position offered, location, education, training, and/or experience. In addition to base pay, also offered is a comprehensive benefits package and 401k contribution (all benefits are subject to eligibility requirements). This position is eligible for participation in a short-term incentive plan subject to the terms of the applicable plans and policies. NOTE: Staffing agencies, headhunters, recruiters, and/or placement agencies, please do not contact our hiring managers directly.