POSITION OVERVIEWNBME's Psychometrics and Data Analysis (PADA) unit is seeking an experienced Senior Psychometrician to join its team. We are particularly interested in candidates with experience or a demonstrated interest in using artificial intelligence and emerging analytical approaches to support operational assessment processes. Candidates should submit a cover letter with their application describing their interest in the position and how their experience aligns with the role.
The Senior Psychometrician will apply advanced measurement expertise to support NBME's operational assessment programs, research initiatives, and continued advancement of psychometric practices. This position will contribute across six primary areas:
- Lead and contribute to applied research focused on enhancing examination design, psychometric methodologies, and analytical procedures.
- Plan, conduct, and evaluate psychometric analyses supporting operational examination programs, including scoring, equating, scaling, and score reporting activities.
- Apply psychometric and measurement expertise to develop and evaluate responsible uses of artificial intelligence, machine learning, and emerging analytical methodologies in operational assessment processes.
- Advance the educational measurement field through professional presentations, publications, and research aligned with both individual and organizational research priorities.
- Assess current psychometric processes and methodologies to identify opportunities, including appropriate uses of AI and automation, to improve the accuracy, efficiency, and effectiveness of examination processing.
- Develop and evaluate data visualizations and graphical displays that clearly communicate assessment information through stakeholder reports, examinee feedback, and interactive data tools.
The Senior Psychometrician will work collaboratively across NBME with professionals in areas such as data analysis, test development, information technology, and program management. The position also involves external communication and collaboration with clients and subject matter experts representing a broad range of medical and health care disciplines.
This role has been designated as primarily remote, with occasional onsite attendance required for team collaboration, meetings, or other business needs. We are prioritizing applicants within a 50-mile radius of our Philadelphia office, while also considering qualified candidates in other states where NBME is authorized to employ staff.
Please note that onsite interviews and onboarding at our Philadelphia office may be required for this role. If applicable, advance notice will be provided to support planning.
QUALIFICATIONSSkills and Abilities
- Strong foundation in psychometric theory and its application to operational assessment, including expertise in item response theory (IRT), classical test theory (CTT), statistical methods, test development practices, and research design.
- Advanced knowledge of measurement methodologies such as equating, scaling, generalizability theory, Bayesian methods, and/or standard setting.
- Demonstrated ability to translate complex assessment results into clear and meaningful graphical displays that support appropriate interpretation, decision-making, and action.
- Knowledge of or experience with natural language processing, process-data analytics, formative assessment methodologies, and related emerging approaches.
- Strong technical and computational skills, including experience using statistical and psychometric software and programming tools such as calibration software, SAS, Python, and/or R.
- Excellent written and verbal communication skills, with the ability to explain technical measurement concepts to audiences with varying levels of psychometric expertise.
- Strong analytical and problem-solving skills, including the ability to identify data, methodological, and operational issues and develop appropriate solutions.
- Excellent interpersonal and collaboration skills, with demonstrated success working with colleagues and stakeholders from diverse professional, technical, and educational backgrounds.
Experience
- A minimum of 5 years of professional experience in a Psychometrician, Measurement Scientist, or comparable applied measurement role is required for consideration. Experience supporting operational testing or assessment programs is strongly preferred.
Education
- A doctoral degree in Educational Measurement, Psychological Measurement, Statistics, or a closely related quantitative field is required.
The compensation we are offering ranges from $138,688 to $173,360. Please note that the offered rate for this position typically aligns with the minimum to midrange of this position, but it can vary based on the successful candidate's qualifications and experience, department budget, and an internal equity review.
The NBME offers competitive salaries, excellent benefits, and a rewarding work environment. Excellent Benefits include: Healthcare, Dental, Prescription, and Vision plans; 401(k) w/match, Tuition Reimbursement Plan, Commuter Benefit: Public Transit or Parking options. Remote Friendly Workplace.
