MULTIPLE POSITIONS AVAILABLE

Employer: AMAZON WEB SERVICES, INC.

Offered Position: Research Scientist III

Job Location: Chicago, Illinois



Position Responsibilities:

Perform all psychometric aspects of exam development, operations, and research including but not limited to automated test assembly, item and test analyses, optimal item bank design, job task analysis, standard setting, quality assurance, and project planning. Conduct all aspects of psychometric analysis in operational work including performing item analysis using psychometric methods, building optimal test forms and pools via optimization techniques, analyzing and monitoring item bank health, setting pass standards via standard setting studies, and supporting Job Task Analysis (JTA) to define and refresh test blueprints. Conduct all aspects of psychometric analysis in developing and applying statistical and psychometric modeling to evaluate and ensure AWS certification exams' validity, reliability, applicability, efficiency, and accuracy. Identify and lead research projects to improve existing operational processes and quality using advanced techniques such as Machine Learning (ML), statistical modeling, Natural Language Processing (NLP), Generative Artificial Intelligence (GenAI), etc. Leverage emerging AI and ML technologies to enhance exam content development, content related management and evaluation, and other aspects of psychometric and exam related tasks. Develop and maintain automation tools using R or Python for psychometric workflow pipeline and other tasks to improve operational efficiencies. Present, interpret, and communicate the results of analyses to stakeholders through written and oral reports. Follow the accreditation standards set by ISO/IEC:2012 17024 and the National Council for Certifying Agencies (NCCA) as they relate to valid psychometric practices. Contribute to the development and execution of the strategic goals regarding the AWS certification program. Consult with leadership, internal staff, external consultants, and industry leaders regarding advancement of current offerings, notably within the AI/ML space. Engage with the professional community through conferences and publications.

Position Requirements:

Ph.D. or foreign equivalent degree in Statistics, Psychometrics, Educational Measurement, Quantitative Psychology, Data Science, Industrial-Organizational (I/O) Psychology, or a related field and three years of research or work experience in the job offered or a related occupation. Must have three years of experience in the following skill(s): (1) working experience in large-scale education, licensure, or certification assessment programs; (2) experience with performing psychometric analyses for large-scale education, licensure, or certification assessment programs with tasks including: (i) item analysis, (ii) equating and scaling, (iii) item response theory and classical test theory, (iv) form and pool assembly, (v) item bank health analysis, (vi) standard setting, or (vii) job task analysis; (3) operational and research experience with at least one advanced test design, such as linear-on-the-fly testing (LOFT) or computerized adaptive testing (CAT); (4) technical knowledge and practical experience in at least one of the following areas: Machine Learning (ML), Natural Language Processing (NLP), or Generative Artificial Intelligence (GenAI); (5) experience in script-based programming skills, specifically R or Python.

40 hours / week, 8:00am-5:00pm, Salary Range $164,697/year to $215,300/year.

Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, visit:

https://www.aboutamazon.com/workplace/employee-benefits.#0000

BASIC QUALIFICATIONS

Position Requirements:

Ph.D. or foreign equivalent degree in Statistics, Psychometrics, Educational Measurement, Quantitative Psychology, Data Science, Industrial-Organizational (I/O) Psychology, or a related field and three years of research or work experience in the job offered or a related occupation. Must have three years of experience in the following skill(s): (1) working experience in large-scale education, licensure, or certification assessment programs; (2) experience with performing psychometric analyses for large-scale education, licensure, or certification assessment programs with tasks including: (i) item analysis, (ii) equating and scaling, (iii) item response theory and classical test theory, (iv) form and pool assembly, (v) item bank health analysis, (vi) standard setting, or (vii) job task analysis; (3) operational and research experience with at least one advanced test design, such as linear-on-the-fly testing (LOFT) or computerized adaptive testing (CAT); (4) technical knowledge and practical experience in at least one of the following areas: Machine Learning (ML), Natural Language Processing (NLP), or Generative Artificial Intelligence (GenAI); (5) experience in script-based programming skills, specifically R or Python.#0000

PREFERRED QUALIFICATIONS

Please see job description and the position requirements above.