Background in cybersecurity or reinforcement learning
Experience in training or fine-tuning models
Practical security judgment from industry or research
Ability to build and debug training experiment code
Responsibilities
Collaborate with cybersecurity researchers to assess context-specific model assistance
Design and implement complex cyber safety evaluations
Research and develop post-training techniques for model safety
Work with the post-training team to integrate safety methods into large model runs
Coordinate with cross-functional teams to uphold safety standards
Benefits
Opportunity to work on cutting-edge AI technology
Collaborative environment with cross-functional teams
Focus on impactful global challenges
Access to professional development resources
Engagement in innovative cybersecurity practices
Full Job Description
About the Role
You'll work directly on the large training runs that shape OpenAI's frontier models. Your focus will be teaching models to help people do legitimate security work while recognizing when their assistance could enable harm. A model that refuses to help a defender investigate a security issue can fail in a different way from one that helps someone misuse it. You'll study both kinds of mistakes and use what you learn to improve the next training run.
We're looking for someone with hands-on machine learning experience who wants to bring security judgment into model training. You might come from cybersecurity and want to go deeper on post-training, or from reinforcement learning and want to build substantial expertise in cybersecurity.
In this role, you will
Work with cybersecurity researchers to identify situations where context changes what help a model should give.
Design complex and realistic cyber safety evaluations.
Research post-training techniques that robustly improves the cyber safety of the frontier models.
Work with the core post-training team to bring promising methods into large model runs and verify that the safety improvements hold at scale.
Coordinate and collaborate with cross-functional teams, including T&S, legal, policy and other research teams, to ensure that our products meet the highest safety standards.
You might thrive in this role if you
Have trained or fine-tuned models yourself and can describe an experiment where you used the results to change how you trained the next model.
Bring practical security judgment from industry work, a research project, collaboration with a security team, or capture-the-flag competitions. We also welcome reinforcement learning practitioners motivated to develop that judgment alongside security researchers.
Can build and debug the code behind a training experiment, from preparing data to checking whether a change in the results reflects a real change in model behavior.
Enjoy working through ambiguous cases with domain experts and translating what you learn into concrete decisions about how a model should respond.
At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.
About OpenAI
OpenAI is an artificial intelligence research laboratory consisting of the for-profit corporation OpenAI LP and its parent company, the non-profit OpenAI Inc. The company was founded in 2015 by a group of technology leaders, including Elon Musk, Sam Altman, Greg Brockman, Ilya Sutskever, and John Schulman. OpenAI's mission is to develop and promote friendly AI for the betterment of humanity. The company has developed a number of cutting-edge AI technologies, including GPT-3, a language processing system that can generate human-like text. OpenAI has received funding from a number of high-profile investors, including LinkedIn co-founder Reid Hoffman and venture capitalist Peter Thiel.