Full Job Description
The Center for Applied Scientific Computing (CASC) within the Computing Directorate is seeking a Postdoctoral Researcher specializing in LLM-based agent to contribute to fundamental R&D on how agents based on large language models (LLMs) are built, evaluated, and used in collaboration with domain experts. Agents are beginning to plan experiments, write and run code, and analyze simulation and experimental data in support of consequential scientific and national security decisions. As a result, the central questions concern the full agentic system rather than the base model alone: how the harness (tools, memory, context management, verification, and guardrails) shapes agent behavior; how domain experts direct, inspect, and correct agents during a task; how benchmarks and environments can measure agent capability in a way that predicts real-world utility; and how post-training can strengthen agentic capabilities in smaller, task-focused models suited for local deployment. Application areas include scientific discovery, national security, autonomous analysis of simulation and experimental data, and agentic workflows for high-performance computing environments. This position will be in the Machine Intelligence Group in the Center for Applied Scientific Computing (CASC) Division within the Computing Directorate at Lawrence Livermore National Laboratory (LLNL).
In this role you will
• Design and build agent harnesses and orchestration frameworks, including tool-calling interfaces, memory and context management, sandboxed code execution, verification and guardrail mechanisms, and multi-agent orchestration, and study how harness design choices affect agent reliability and behavior.
• Build benchmarks and task environments for agentic systems in scientific and national security settings, including task specification, verifiable or expert-grounded success criteria, and analysis of where existing benchmarks fail to predict real-world performance.
• Develop human-agent collaboration workflows in which domain experts set goals, review intermediate reasoning and actions, and correct or redirect agents, and study how to use expert feedback to improve agent capabilities.
• Establish rigorous evaluation methodology for agentic systems, including trajectory-level analysis, failure-mode characterization, and assessment of robustness, safety, and reproducibility.
• Apply post-training methods (e.g., supervised fine-tuning, preference optimization, reinforcement learning with verifiable rewards) to improve agentic capabilities such as tool use, long-horizon planning, and self-verification, including in smaller models suited for local deployment.
• Collaborate with project scientists and engineers to formulate research on complex problems stemming from national security and scientific applications.
• Document results in peer-reviewed publications and technical reports, contribute to proposals, and pursue independent but complementary research interests within a multidisciplinary team.
• Perform other duties as assigned.
Qualifications
• Ph.D. in computer science, machine learning, human-computer interaction, applied mathematics, statistics, or a related field.
• Demonstrated research experience in one or more of the following areas: LLM agents and tool use, agentic system design and evaluation, benchmark or environment construction for LLMs, human-machine collaboration and interaction, or LLM post-training.
• Experience developing or applying deep learning methods using modern libraries such as PyTorch, including experience with LLM inference and/or training frameworks.
• Demonstrated research productivity, as documented by publications, reports, presentations, and/or open-source software in relevant venues (e.g., NeurIPS, ICML, ICLR, ACL, EMNLP, COLM, AAAI, CHI).
• Strong software engineering skills in the Python ecosystem.
• Demonstrated ability to acquire substantial domain knowledge in fields of application and communicate effectively with subject matter experts from other fields.
Qualifications We Desire (optional)
• Hands-on experience building agent harnesses or agentic workflows, including tool integration, sandboxed execution, memory and context management, verification and guardrails, and multi-agent orchestration.
• Experience designing and releasing benchmarks, task environments, or evaluation suites for LLMs or agents, including defining verifiable or expert-grounded success criteria and validating that benchmark performance reflects downstream utility.
• Experience studying or designing human-machine collaboration, such as mixed-initiative systems, learning from human feedback and corrections, expert oversight of autonomous systems, or user studies of LLM-assisted workflows.
• Experience with LLM post-training, synthetic data generation, self-improvement pipelines, or test-time compute scaling for agentic capabilities.
• Experience with high-performance computing, distributed or large-scale training, and/or scientific machine learning applications such as simulation workflows, experimental data analysis, or scientific code generation.
Pay Range
$143,328 Annually
Position Information
This is a Postdoctoral appointment with the possibility of extension to a maximum of three years. The postdoctoral appointment is a temporary, mentored training position intended for individuals in the early stages of their postdoctoral research careers. Consistent with the structure of the program, applicants must have received their Ph.D. (or equivalent doctorate) within five years of the anticipated start date.
Security Clearance
None required. However, if your assignment is longer than 179 days cumulatively within a calendar year, you must go through the Personal Identity Verification process. This process includes completing an online background investigation form and receiving approval of the background check.