5-7 years of experience in machine learning, specifically with large language models (LLMs)
Strong background in post-training techniques and evaluation methodologies
Experience in subjective domains such as design, writing, or visual style
Ability to think creatively and work in ambiguous environments
Startup experience preferred, showcasing adaptability and passion for innovation
Responsibilities
Train reward models and classifiers for subjective evaluation
Develop evaluations and benchmarks tailored for creative domains
Conduct post-training experiments on open-source models
Establish and maintain infrastructure for ML experiments
Collaborate with AI labs and creative professionals on experimental design
Manage the entire ML pipeline from experimentation to publication
Publish findings through blogs and whitepapers
Benefits
Opportunity to work on cutting-edge AI projects
Collaborative environment with AI labs and creative experts
Focus on innovative and subjective evaluation methods
Chance to influence the future of AI in creative fields
Engagement in research and publication opportunities
Full Job Description
About the role
You'll own full-stack ML experiments, training runs, infra, data and eval pipelines, and publishing results. You'll develop Taste's internal research, evaluation design, judges and reward models while collaborating directly with AI labs on frontier projects.
What You'll Do
Train reward models, classifiers, judges, and verifiers for subjective domains (e.g. design, writing, visual style).
Develop frontier evaluations and benchmarks for subjective domains.
Run post-training experiments on open-source models to test new data formats and post-training techniques.
Setup the infrastructure to run this.
Collaborate with AI labs and creative experts to design pilots and experiments around taste.
Own the end to end pipeline.
Publish blogs and whitepapers.
You Might Be a Good Fit If You
Are obsessed with taste and want a world with less AI slop.
Experience in post-training and in evals/judges
LLM experience required, cannot be purely classical ML
Thinks like a researcher, moves like an engineer. Creative, scrappy, comfortable in ambiguity
Startup experience strongly preferred; deep passion for creativity and design
