Manager, Assessment Development & InnovationLocation: Lake Mary, FL (Hybrid)
Department: Global Certifications
Status: Full-Time | Exempt
Position SummaryThe Institute of Internal Auditors (IIA) is seeking a
Manager, Assessment Development & Innovation to support and advance our certification exam programs through rigorous, standards-aligned test development practices.
This role plays a central part in job task analysis (JTA), content development, item review, and standard-setting activities that help ensure certification exams remain valid, defensible, and reflective of current practice in the field.
The Manager will also lead independent initiatives exploring new assessment methods, innovative content development, and emerging technologies-including AI-assisted item generation and scoring-to strengthen and modernize The IIA's portfolio of certification programs.
Success in this position requires strong project management skills, sound psychometric judgment, analytical capabilities, and the ability to effectively facilitate and collaborate with subject matter experts (SMEs), volunteer committees, external stakeholders, and cross-functional internal teams.
Work ArrangementThis is a
hybrid position based in Lake Mary, Florida.
What You'll Do- Facilitate SME and stakeholder sessions throughout the exam development lifecycle, including JTA data-gathering sessions, item review panels, and standard-setting sessions using Modified Angoff or comparable methodologies.
- Develop interview protocols and conduct focus groups to support JTA data collection.
- Design and implement JTA surveys to gather practitioner input at scale.
- Develop PowerPoint presentations, pre-meeting materials, and other resources that support SME understanding of JTA, item curation and review, and standard-setting activities.
- Prepare and organize data for standard-setting activities, including item-level statistics and rating materials.
- Manage end-to-end SME item curation, including assignment, tracking, and quality control.
- Evaluate AI-generated exam items for accuracy, relevance, quality, and alignment with content specifications.
- Analyze survey and standard-setting data to validate task and knowledge statements, inform syllabus development, and confirm consistency and defensibility.
- Develop clear, accurate reports summarizing JTA methodology, findings, and recommendations for internal and external stakeholders.
- Write standard-setting reports suitable for accreditation and legal defensibility, including alignment with ISO/IEC 17024 and applicable Uniform Guidelines.
- Lead new assessment initiatives with minimal oversight, exercising sound judgment and proactive project management.
- Collaborate on or lead market research to benchmark exam development practices, emerging credentialing trends, and competitor programs.
- Design and test prototypes for new exam item types and formats.
- Explore and evaluate AI-based scoring approaches for open-response and constructed-response items, assessing feasibility, validity, and fairness implications.
- Collaborate closely with SMEs, volunteer committees, and external stakeholders, including chapters, Institutes, and Education programs.
- Partner with internal Certification, Education, Marketing, and other cross-functional teams.
- Coordinate, as needed, with external vendors supporting AI tools, testing platforms, or standard-setting logistics.
- Manage multiple concurrent projects and competing deadlines while maintaining quality, accuracy, and alignment with program objectives.
QualificationsRequired Qualifications- Bachelor's degree in Psychology, Measurement, Education, or a related field, or equivalent professional experience.
- 3-5+ years of experience in assessment/test development, certification, or credentialing programs.
- Demonstrated experience with job task analysis, item writing and review, and standard-setting methodologies.
- Strong facilitation skills with the ability to effectively guide SME panels toward consensus.
- Excellent written communication skills, including experience producing clear technical reports for varied audiences.
- Proficiency with survey design and data analysis tools, including Excel.
- Strong analytical and data interpretation skills.
- Strong project management and organizational skills with the ability to manage multiple concurrent projects and competing deadlines.
- Ability to work independently, exercise sound judgment, and lead initiatives with minimal oversight.
- Ability to work effectively with SMEs, volunteer committees, external stakeholders, and cross-functional teams.
- Curiosity, adaptability, and willingness to explore emerging assessment technologies, including AI-assisted item generation and automated scoring.
Preferred Qualifications- Advanced degree in Psychology, Psychometrics, Measurement, or a related field.
- Familiarity with ISO/IEC 17024 and other accreditation standards relevant to certification bodies.
- Experience evaluating or working with AI-generated content in a testing or content development environment.
- Exposure to AI or automated scoring approaches for constructed-response items.
- Experience with item banking or exam authoring platforms.
- Experience with statistical software or qualitative analysis tools.
Core Competencies- Psychometric and measurement judgment
- Project management and self-direction
- SME and stakeholder facilitation
- Analytical and data interpretation skills
- Clear technical writing
- Curiosity and adaptability toward emerging testing technologies, including AI-assisted item generation and automated scoring
