Member of Technical Staff, Embedded Assessments

METR

$402K — $500K+*
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • At least 2 of the following: skilled in LLM prompting and relevant research or a quick learner in these areas.
  • Some experience in security.
  • Ability to rapidly familiarize with large codebases.
  • Creative, self-directed, and scrappy when it comes to problem-solving during exercises.
  • Expertise in loss-of-control threat modeling and safety analysis.
  • Strong verbal communication, writing, and stakeholder management skills.
  • Trustworthy with a solid reputation.

Responsibilities

  • Conduct embedded exercises at a frontier AI lab for several weeks.
  • Develop and standardize methodologies for assessments.
  • Collaborate and communicate with researchers to enhance exercise outcomes.
  • Build tools to improve future assessment exercises.
  • Assist with hiring and onboarding to scale the team.
  • Document results and coordinate future assessment planning.

Benefits

  • Catered lunch and dinner daily with access to an in-office gym and shower.
  • Relocation stipend for moving to the Bay Area.
  • Unlimited PTO and 21 weeks of parental leave for new parents.
  • Monthly transit/parking stipend and annual Uber budget.
  • Professional development funding for courses, training, and conferences.
  • Mental health support for therapy and treatment expenses.
  • Wellness benefits for gym and health-related expenses.
  • Reimbursement for home office setup and equipment.
Full Job Description
Overview

METR has started embedding researchers inside frontier labs to investigate incidents, stress-test labs' internal agent monitoring systems, and assess loss-of-control risks from internal deployment. As agent capabilities increase, we expect this to be one of the most important sources of independent information the world has about catastrophic risks from advanced AI.

As the source of information becomes more important, we'll need many more talented researchers and engineers who can conduct embedded exercises. We expect these assessors to have deep access, and for their work to be a large part of METR's impact in the next year. We want to build on the momentum from previous exercises to further develop our risk assessments.

What the job looks like

You'll be embedded in a frontier AI lab for up to several weeks at a time, likely alongside 1-4 other METR staff. Between exercises, you'll practice, develop the general methodology, talk to other researchers, build tooling to make future exercises go better, help us hire and scale, write up results, and plan/coordinate future exercises.

We are looking for embedded researchers and engineers across multiple current and potential future exercises: AI R&D acceleration assessment, compute allocation, monitorability red-teaming, incident investigations, and more. We expect this role to evolve significantly as we develop and prototype this new form of risk assessment.

Ideal candidate

We are looking for candidates with at least two of the following skills, although the ideal candidate will have most or all of them:
  • You're good at prompting LLMs, are familiar with research relevant to the fields above (or can get up to speed quickly).
  • You have some security experience.
  • You can get spun up on large codebases quickly.
  • You're scrappy, creative, independent, and self-directed (because during the exercises you'll only have a few other METR employees you can talk to). The work is novel and you'll need to figure a lot of stuff out on the fly largely by yourself.
  • You are excellent at loss-of-control threat modeling and breaking down safety cases.
  • You're good at verbal communication, writing, and stakeholder management (e.g., navigating complex relationships with frontier labs).
  • You're trustworthy and have a good reputation.


Compensation & Benefits

$402,048 - $687,759 a year

METR also has a host of benefits:
  • The office: Catered lunch and dinner daily; in-office gym and shower
  • Relocation support: Stipend for moving to the Bay Area
  • Time-off and leave: Unlimited PTO and 21-week parental leave for new parents
  • Commuter benefit: Monthly transit/parking stipend and an annual Uber budget
  • Professional development benefit: for training, courses, conferences, and AI safety education
  • Mental health benefit: for therapy, medication, and other mental health expenses
  • Wellness benefit: for gym memberships and other wellness expenses
  • Work equipment benefit: for home office and workstation equipment expenses


Hybrid Preferred: Our technical team members are in our office in Berkeley 3-5 days/week. We would ideally like for you to be in person too, but we are happy to be flexible here. If you lack US work authorization and would like to work in-person, we can likely sponsor a cap-exempt H-1B visa for this role.

We encourage you to apply even if your background may not seem like the perfect fit! We would rather review a larger pool of applications than risk missing out on a promising candidate for the position.

Similar Jobs

More Jobs at METR

More Information Technology Jobs

Find similar Member of Technical Staff, Embedded Assessments jobs: