This Principal-level role serves as a senior technical evaluation and AI governance expert within a multidisciplinary team supporting the UHC Clinical AI Governance Council. This individual contributor role is responsible for ensuring that AI-enabled clinical products and use cases are appropriately tested, evaluated, and documented prior to implementation and that evidence presented to the Council is rigorous, reproducible, and aligned with clinical risks, performance metrics, AI safety considerations, and defined success thresholds.
The position owns the evaluation framework and evidence standards used to support AI vendor vetting and clinical AI review. The role ensures that each use case is evaluated against clearly defined risks, metrics, and thresholds to enable objective and consistent Council decision-making. The role advises on and reviews evaluation strategies for clinical AI use cases and, when needed, conducts independent assessments to validate performance claims, testing results, and evidence submitted for Council review.
The role serves as the primary technical and methodological liaison among Council leadership, program consultants, and subject matter experts across clinical, data science, engineering, Legal, and Compliance domains. The position provides independent evaluation of AI-enabled solutions and translates complex AI concepts and evidence into clear, decision-ready recommendations for senior clinical leadership.
You'll enjoy the flexibility to work remotely* from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week.
Primary Responsibilities: - Partner with Council leadership and program consultants to apply the submission framework to each use case, ensuring analytic components align the AI's intended use and defined risks, metrics, and thresholds
- Define use case-specific measurable performance, success criteria, translating clinical risks into measurable metrics and thresholds for decision-making
- Ensure complete and clearly defined evaluation methods at intake, including data, populations, comparators, metrics, test methods, and study design
- Serve as the primary technical liaison across data science, clinical, and engineering SMEs, synthesizing inputs into a consistent evaluation approach
- Ensure that evaluation results support clear decisions against defined performance and success thresholds
- Develop concise, decision-ready summaries of performance against defined thresholds, including key limitations of the AI supported program and AI evaluation methodology
- Serve as a senior methodological resource during Council review, addressing questions on evaluation design and results
- Integrate analytic, clinical, and operational inputs into a single, aligned recommendation
- Define requirements for post-deployment monitoring, including metrics, thresholds, accountable governance groups, and triggers for re-review
- Design and execute analytic evaluations of clinical AI use cases using real-world healthcare data
- Perform hands-on data programming, analysis, and interpretation to support Council reviews and broader evaluation initiatives
You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.
Required Qualifications: - 5+ years of experience working with healthcare data, healthcare technology, clinical operations, healthcare analytics, or related healthcare domains
- 5+ years of experience using Python, SQL, R, or similar tools to develop, test, evaluate, or validate analytic, machine learning, or AI-enabled solutions
- 5+ years of experience working across clinical, technical, operational, data science, or product teams to translate complex technical concepts into business or clinical recommendations
- 3+ years of hands-on experience developing, implementing, testing, or evaluating AI solutions, including predictive models, NLP systems, LLM-based applications, chatbots, retrieval-augmented generation (RAG) systems, or agentic workflows
- 3+ years of experience designing, conducting, or overseeing testing, validation, benchmarking, or performance assessment of AI-enabled solutions, including evaluation of accuracy, safety, reliability, bias, limitations, or clinical appropriateness
Preferred Qualifications:- Experience developing or deploying LLM-based applications, chatbots, retrieval-augmented generation (RAG) systems, agentic AI workflows, or AI-enabled clinical decision-support tools
- Experience evaluating AI solutions in healthcare settings, including assessment of accuracy, safety, bias, reliability, workflow integration, or clinical appropriateness
- Experience supporting AI governance, model risk management, responsible AI, algorithm oversight, or technology review committees
- Experience conducting vendor assessments, proof-of-concept testing, pilot evaluations, or technical due diligence for AI-enabled products
- Experience presenting complex technical findings to senior leaders, clinicians, or governance bodies and translating results into actionable recommendations
- Experience reviewing, designing, or overseeing test plans for AI-enabled products in healthcare or other regulated environments
*All employees working remotely will be required to adhere to UnitedHealth Group's Telecommuter Policy
Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you'll find a far-reaching choice of benefits and incentives. The salary for this role will range from $112,700 - $193,200 annually based on full-time employment. We comply with all minimum wage laws as applicable.
