About the RoleIn this role, you'll own the synthetic data pipeline end-to-end: transforming domain-specific workflows into structured, realistic, and challenging training tasks for AI agents. You'll work directly with subject-matter experts, design generation and validation systems, and develop the metrics that tell us whether synthetic tasks are actually teaching models what we want.
What You'll Do- Build and maintain the synthetic data pipeline, turning domain-specific workflows into realistic, structured, and challenging training tasks for AI agents.
- Collaborate with subject-matter experts across professional and technical domains to design high-quality synthetic tasks.
- Design synthetic task generation methods that produce diverse, realistic, and learnable data at scale.
- Build tooling to mutate, validate, and iteratively improve synthetic tasks.
- Analyze model and agent performance on synthetic tasks to understand what they teach and where they break down.
- Develop metrics to quantify synthetic task diversity, realism, learnability, and overall quality.
What We're Looking ForRequired- 2-4 years of relevant engineering experience.
- Proficiency in Python, Docker, and Linux environments.
- Hands-on experience with synthetic data research methods.
- Strong intuition for what makes synthetic data "good" - and an honest understanding of its limitations.
- Demonstrated ability to build synthetic data pipelines end-to-end without a fully prescribed roadmap.
- Experience working with environments, evaluations, and benchmarks.
- Detail-oriented mindset for spotting subtle inconsistencies and edge cases in synthetic data.
- Ability to reason from first principles about task design, scoring functions, and failure modes.
- Comfort thriving in unstructured, early-stage environments where you define the path forward.
- Strong written and verbal communication skills for async, cross-timezone collaboration.
Nice to Have- Background in reinforcement learning or post-training data for large language models.
- Experience building reward signals, graders, or automated QA systems for agent tasks.
- Prior work at an early-stage AI or ML startup.
Compensation & Benefits- Salary: $150,000 - $250,000 USD annually, depending on experience.
- Visa sponsorship: Available.
- Equity participation in a well-funded, early-stage AI company.
LocationThis is an
on-site role based in
San Francisco, CA. We work together in person - candidates should be prepared to be in the office regularly. Remote arrangements are not available for this position.
