IQVIA

Senior AI Scientist

IQVIA • $108K — $270K *
Durham, NC 27713In-Person
Information Technology
5 - 7 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • Master's or PhD in Computer Science, Artificial Intelligence, Machine Learning, or related field
  • 5+ years of experience in applied AI/ML with focus on Generative AI and LLMs
  • Demonstrated expertise in LLM evaluation including automated and human-in-the-loop methods
  • Hands-on experience with RAG systems, embeddings, and agent frameworks
  • Strong understanding of evaluation metrics, experimental design, and debugging techniques

Responsibilities

  • Design and implement scalable evaluation frameworks for LLMs across various use cases
  • Develop automated and human-in-the-loop evaluation pipelines
  • Define and monitor critical performance metrics such as accuracy and latency
  • Establish evaluation strategies for agent-based and multi-agent systems
  • Implement offline and online evaluation mechanisms, including A/B testing
  • Ensure compliance with Responsible AI principles and regulatory requirements
  • Build and scale evaluation tooling and integrate with enterprise AI platforms

Benefits

  • Opportunities for mentorship and professional development
  • Exposure to cutting-edge AI technologies in regulated environments
  • Collaboration with cross-functional teams from various domains
  • Contribution to strategic AI initiatives impacting healthcare and life sciences
Full Job Description
Role Overview

We are seeking a Senior AI Scientist to lead the design, development, and operationalization of evaluation frameworks for Generative AI systems, with a primary focus on Large Language Models (LLMs) and agentic AI solutions.

This role will be responsible for defining and implementing robust methods to assess quality, safety, reliability, and business impact across LLM-powered applications and multi-agent workflows. The position operates within regulated environments such as life sciences, clinical research, and regulatory domains, ensuring that AI systems meet enterprise and compliance standards.
Key Responsibilities
1. LLM Evaluation & Benchmarking
  • Design and implement scalable evaluation frameworks for LLMs across use cases including:
    • Question answering, summarization, information extraction, and reasoning
    • Clinical and regulatory document generation (e.g., ICFs, CSRs, protocols)
  • Develop both automated and human-in-the-loop evaluation pipelines
  • Define, measure, and monitor key performance metrics, including:
    • Accuracy, factuality, faithfulness, and hallucination rate
    • Robustness, consistency, latency, and cost-performance trade-offs
  • Build domain-specific benchmarks using real-world clinical, regulatory, and RWD data
2. Agentic AI & Multi-Agent Evaluation
  • Establish evaluation strategies for agent-based and multi-agent systems
  • Measure and analyze:
    • Task completion success rates
    • Planning and reasoning quality
    • Tool usage accuracy
    • Inter-agent coordination and failure patterns
  • Develop scenario-based and simulation-driven evaluation environments
  • Evaluate orchestration frameworks (e.g., LangGraph, Semantic Kernel, Claude Agents)
3. End-to-End System Evaluation
  • Define evaluation strategies for complete AI pipelines, including:
    • Retrieval-Augmented Generation (RAG) systems
    • Tool-augmented agents
    • Knowledge graph + LLM architectures
  • Implement offline and online evaluation mechanisms, such as:
    • A/B testing and canary releases
    • Production monitoring and model drift detection
  • Enable observability and traceability using tools such as LangSmith and OpenTelemetry
4. Responsible AI & Compliance
  • Ensure all evaluation practices align with Responsible AI principles and regulatory requirements (e.g., GxP)
  • Assess and mitigate risks related to:
    • Bias, fairness, safety, and explainability
    • Data leakage and privacy concerns
  • Develop audit-ready evaluation frameworks suitable for regulated environments (e.g., FDA, EMA)
5. Tooling & Platform Development
  • Build and scale evaluation tooling, including:
    • Automated evaluation pipelines
    • Prompt and version tracking systems
    • Experiment management platforms
  • Integrate evaluation frameworks with enterprise AI platforms (e.g., Azure, Databricks, AWS, on-prem GPU environments)
6. Leadership & Collaboration
  • Collaborate with cross-functional teams including:
    • AI/ML engineers, product teams, and domain scientists
    • Clinical, regulatory, and real-world evidence stakeholders
  • Establish enterprise-wide evaluation standards and best practices
  • Mentor junior team members and contribute to strategic AI initiatives
Required Qualifications
  • Master's or PhD in Computer Science, Artificial Intelligence, Machine Learning, or a related field
  • 5+ years of experience in applied AI/ML, with a strong focus on Generative AI and LLMs
  • Demonstrated experience in:
    • LLM evaluation (both automated and human-in-the-loop)
    • Prompt engineering and model behavior analysis
    • Python programming using frameworks such as PyTorch or TensorFlow
  • Hands-on experience with:
    • RAG systems, embeddings, and vector databases
    • Agent frameworks (e.g., LangChain, LangGraph, Semantic Kernel)
  • Strong understanding of:
    • Evaluation metrics and experimental design
    • Model limitations, failure modes, and debugging techniques
Preferred Qualifications
  • Experience in life sciences, healthcare, or regulated environments
  • Familiarity with:
    • Clinical trial workflows (ICF, CSR, TMF, regulatory submissions)
    • Knowledge graphs and biomedical data systems
  • Experience with evaluation tools such as LangSmith, Promptfoo, DeepEval, HELM, or OpenAI Evals
  • Exposure to Responsible AI frameworks and regulatory compliance standards
What Success Looks Like
  • Standardized evaluation frameworks adopted across AI teams
  • Measurable improvements in LLM reliability and agent performance
  • Defined quality gates to support production-ready AI deployment decisions
  • Strong alignment between AI model performance and business/clinical outcomes


The potential base pay range for this role, when annualized, is $108,000.00 - $270,000.00. The actual base pay offered may vary based on a number of factors including job-related qualifications such as knowledge, skills, education, and experience; location; and/or schedule (full or part-time). Dependent on the position offered, incentive plans, bonuses, and/or other forms of compensation may be offered, in addition to a range of health and welfare and/or other benefits.

About IQVIA

Innovex offers superior and creative advice to corporations. In addition, they raised venture capital for our own affiliate companies, in addition to having assisted clients in fund-raising efforts, new venture outreaches, and financing. Their engineers and technical staff offer varied backgrounds in manufacturing, high tech, product design, quality control, safety, biomechanics, and robotics.

IQVIA Careers

Join the vibrant team at IQVIA, a leading global provider of advanced analytics, technology solutions, and clinical research services to the healthcare industry. IQVIA is recognized for its commitment to innovation, leadership in healthcare, and a culture that celebrates diversity and inclusion.

Work You’ll Do

At IQVIA, we offer job opportunities that empower you to leverage your skills and drive healthcare forward. Become part of a team that thrives on innovation and the pursuit of operational excellence, helping to improve patient outcomes across the globe.

Transform Your Career

IQVIA is the place where you can transform the world of healthcare while advancing your career. With a variety of positions available, from research and development to business analytics, IQVIA is at the forefront of the industry. Our leadership is committed to fostering a culture of growth and professional development, making IQVIA a prime environment for both seasoned professionals and those seeking an internship to kickstart their career.

Innovative Work Environment

Join over 50,000 dedicated professionals in a dynamic environment where technology meets healthcare. At IQVIA, innovation isn’t just a buzzword; it's the backbone of how we solve complex challenges. With cutting-edge resources and a collaborative global team, your work at IQVIA directly contributes to advancements in life sciences.

Career Development and Benefits

IQVIA believes in nurturing its team members not just as employees but as individuals seeking to grow their careers. We support this with unmatched benefits, diversity training, and opportunities for leadership development. Our commitment to your career growth is matched by our dedication to providing a supportive and inclusive workplace.

Explore Job Opportunities

Whether you’re looking for a full-time position, part-time work, or a graduate internship, IQVIA offers a range of employment options to match your career ambitions. Our hiring process is designed to be transparent and engaging, helping you to understand where your skills can best be utilized within the company.

Networking and Professional Development

At IQVIA, networking and professional development are part of our DNA. Engage with industry leaders, participate in global conferences, and take advantage of our in-house training programs designed to enhance your skills and advance your career.

Stay Connected

Join Our Team Search open positions that match your skills and interests. We look for passionate, curious, creative, and solution-driven team players. Explore the diverse job opportunities at IQVIA and find where you can make an impact.

Keep Up to Date

Stay ahead with career tips, insider perspectives, and industry-leading insights you can put to use today—all from the people who work here.

Job Alert Emails

Personalize your subscription to receive job alerts, latest news, and insider tips tailored to your preferences. Discover the exciting and rewarding career opportunities that await at IQVIA. Embark on a journey of growth, innovation, and leadership at IQVIA. Shape your future in an environment that values diversity and fosters professional development. Join IQVIA today and transform healthcare through the power of information and technology.
Learn more about IQVIA
Size
82,000 employees
Market Cap
$37.5 billion
Industry
Pharmaceuticals & Biotech
Net Income
$279 million
Founded
2016
5 Year Trend
+15.3%
Revenue
$11.3 billion
NASDAQ
IQV
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at IQVIA

More Information Technology Jobs

Find similar Senior AI Scientist jobs:

NationwideDurham, NC