Title: Healthcare AI Evaluation and Risk Assessment SpecialistJob Type: Full Time Employee
Schedule: Monday through Friday (no Holidays)
Location: This position is 100% telework
Digital Health Center of Excellence (DHCoE), Office of Strategic Partnerships and Technology Innovation (OST), Center for Devices and Radiological Health (CDRH), FDAPosition SummaryThe Healthcare AI Evaluation and Risk Assessment Specialist provides scientific, technical, and regulatory expertise supporting FDA's oversight of AI-enabled medical devices. The incumbent serves as a subject matter expert in the evaluation of emerging healthcare AI technologies, including generative AI, large language models (LLMs), foundation models, multimodal AI systems, and agentic AI, and their application in medical devices across clinical specialties. The position focuses on evaluating safety, effectiveness, performance, reliability, risk management, and evidence supporting AI-enabled medical devices and contributing to regulatory science, policy development, and review best practices. [fda.gov], [fda.gov]
Major Duties and Responsibilities- Review premarket submissions, IDE applications, and other regulatory submissions involving AI-enabled medical devices
- Evaluate AI model performance, validation strategies, benchmarking methodologies, and supporting evidence for safety and effectiveness
- Assess AI risk assessment and risk management approaches, including model failure modes, hallucinations, bias, automation bias, cybersecurity risks, human factors, post-market monitoring, and lifecycle management
- Evaluate model testing approaches, including benchmarking, robustness testing, subgroup analyses, uncertainty quantification, stress testing, and adversarial testing (red teaming)
- Identify scientific limitations of proposed evaluation designs, datasets, scoring methodologies, and validation approaches
- Assess evidence from clinical studies, reader studies, simulation-based evaluations, real-world evidence, and external validation studies
- Provide scientific consultation to multidisciplinary review teams on AI evaluation methodologies, evidence generation, transparency, explainability, governance, and performance monitoring
- Collaborate with FDA staff, other Federal agencies, academic institutions, healthcare organizations, standards development organizations, and industry stakeholders
- Contribute to the development of regulatory science, policy, guidance documents, review methodologies, and best practices related to healthcare AI and emerging digital health technologies
- Monitor advances in AI technologies and identify scientific challenges and opportunities relevant to FDA's regulatory mission
- Prepare technical reports, scientific reviews, briefing materials, presentations, and policy analyses
Minimum Qualifications- Master's degree or higher in artificial intelligence, machine learning, computer science, data science, biomedical engineering, health informatics, medicine, public health, or a related field
- At least seven (7) years of relevant experience in healthcare AI, medical devices, digital health, regulatory science, or related disciplines
- Demonstrated expertise in AI evaluation, risk assessment, evidence generation, and performance assessment methodologies
Preferred Qualifications- PhD, MD, DVM, PharmD, or other advanced scientific degree
- Experience supporting FDA or other Federal healthcare agencies
- Knowledge of:
- Generative AI and LLMs
- Foundation and multimodal models
- Software as a Medical Device (SaMD)
- Medical device regulatory pathways (510(k), De Novo, PMA, IDE)
- AI governance and Good Machine Learning Practice (GMLP)
- Medical device cybersecurity and real-world evidence
Must be a US Citizen or Full Green Card holder.
Please be advised that certain client contracts may outline specific requirements for onsite work. Tunnell employees and/or contractors are expected to comply with all client requirements, including any safety and health protocols.
