4+ years of experience in ML or software engineering, preferably in AI labs or product companies.
Hands-on experience with fine-tuning or post-training large language models (LLMs).
Proven track record in designing evaluations for product release decisions.
Expertise in LLM red teaming, browser agents, model routing, or production agents with tools.
Strong proficiency in Python and familiarity with ML training tools; TypeScript experience is a plus.
Desire to take a hands-on role in shaping technical direction and team growth.
Responsibilities
Build evaluation harnesses using model judges and human preference data.
Post-train models through fine-tuning and ship results to production.
Develop model routing strategies to optimize answer quality and cost.
Enhance agent performance on websites lacking APIs, managing user interactions effectively.
Identify risks like prompt injection through adversarial testing.
Advance memory and retrieval systems for improved decision-making.
Analyze real conversations to inform model improvements.
Benefits
Opportunity to join a founding team at a Y Combinator-backed startup.
Hands-on role with significant influence on product direction.
Work in a dynamic, innovative environment focused on AI advancements.
Collaborate with a passionate team dedicated to improving AI reliability.
Full Job Description
About the Role
Join the founding team at a Y Combinator-backed AI assistant startup to build the models and systems behind a useful, reliable AI agent. You will lead hands-on research and engineering that improves answer quality while managing latency, cost, and safety, and ship improvements to users regularly. What You'll Do
Build evaluation harnesses using model judges, human preference data, and regression tests to guide release decisions.
Post-train models through supervised fine-tuning, preference tuning, and distillation, then ship the results to production.
Develop model routing that balances answer quality, latency, and cost across smaller and frontier models.
Improve agent performance on websites without APIs, including navigation, sign-in, and checkout, with appropriate escalation when needed.
Identify and address prompt injection, data leakage, and other risks through adversarial testing and evaluations.
Advance memory, retrieval, tool selection, and decisions about when the agent should act or ask for help.
Review real conversations and turn findings into evaluation and model improvements.
What We're Looking For
At least 4 years of ML or software engineering experience at an AI lab, AI product company, or similar organization.
Hands-on experience fine-tuning or post-training LLMs and deploying the results to production.
Experience designing evaluations that teams have used to make product release decisions.
Background in at least one area: LLM red teaming, browser or computer-use agents, model routing, or production agents with tools and persistent memory.
Strong Python skills and experience with ML training and evaluation tools, plus TypeScript experience in a product codebase.
A hands-on, founding-team approach, with interest in setting technical direction and helping grow the team.
Compensation & Benefits
Salary range: $200,000 to $300,000 USD annually. Visa sponsorship is not available. Location
On-site in San Francisco, California, United States.
