Description

PREP Research Associate

This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.

Research Title:

Measuring Accuracy Uplift From LLM Adversarial Pressure/AI Research

The work will entail:

Overview: ITL's AI Program role in the project "Measuring Accuracy Uplift From LLM Adversarial Pressure/AI Research" involves the following tasks: (1) developing and evaluating an agentic deep research pipeline in which large language models (LLMs) autonomously research a corpus of authoritative documents and produce a fully cited report, and (2) designing adversarial LM-judge evaluation probes that measure whether the pipeline's claims and citations are factually grounded in the source material. Our success depends upon the availability of highly skilled domain experts. We are challenged with difficult tasks that require not only expertise in agentic AI systems and large language models, but also in designing evaluation methodologies for applications where measuring factual grounding under adversarial pressure has not been attempted before.

U.S. Citizen Preferred

Key responsibilities will include but are not limited to:

§ Developing and tuning LM-judge evaluation probes that measure the factual grounding and citation quality of agentic AI outputs.

§ Design adversarial pressure scenarios that stress-test an agentic research pipeline's factual accuracy against a trusted document corpus.

§ Analyze the accuracy uplift attributable to adversarial verification and compare that approach against baseline agentic pipeline performance.

§ Produce high-quality publications based on research and results present at internal and external meetings and conferences.

Qualifications

§ US citizenship is preferred.

§ An M.S. or PhD degree in Computer Science with 3 or more years of relevant experience.

§ Expertise in Python and state of the art AI models, including large language models and agentic AI systems.

§ Ability to build deployable complex software solutions for agentic AI evaluation pipelines.

§ Strong oral and written communication skills and strong presentation skills.

Application Instructions

Please upload the following with your application:
• CV/Resume

*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender