Full Job Description
You9ll work with large enterprises to capture their data and transform it into high-fidelity RL environments for capability evaluations and training datasets for frontier labs. We focus on pushing the frontier of world-building, verifier engineering, and more alongside our partners.
Your goal will be to automate the process of building evals for real work in the economy.
What You9ll Do
3 Ship models for workflow extraction, classification, and grading.
3 Engineer autonomous task refinement processes which distill data taste into pipelines.
3 Deliver data to customers and deploy into real engagements.
3 Help define the future of agentic transformation for enterprises around the world.
3 Deeply learn about the intricacies of enterprises through building evaluations for all aspects of work.
3 Build end-to-end environments for labs & enterprises by platformizing sandbox app clones, load real data into the sandboxes, build prompts from real workflows, and write verifiers leveraging enterprise expertise & golden outputs.
3 Systematize the production of environments to scale throughput while maintaining high-quality worlds and verifiers.
What We9re Looking For
3 Prior experience shipping environments - you9ve contributed to an OSS framework, built environments at previous companies, or worked on agentic evaluations.
3 Strong full-stack engineering skills - you9ll be responsible for everything from infrastructure to app code to analytics
3 Bias to action - this team is focused on shipping evals, not just philosophizing about them.
3 Curiosity - being biased towards understanding and digging deep into model behavior and actually looking at the data.
3 Sweat the details that make a simulation indistinguishable from the real thing and have systems-level thinking skills that allow you to scale up quality.
Nice to Have
3 Experience with Temporal, Modal, or similar orchestration/compute services
3 Experience with synthetic data generation for frontier models.Past work auditing and scrutinizing industry-standard evaluations
Benefits
3 Semi-annual performance bonus structure
3 Generous equity grant vested over 4 years
3 Up to $15k Relocation bonus
3 $10K housing bonus (if you live within 0.5 miles of our office)
3 $1.5K monthly stipend for meals
3 Free Equinox membership
3 $200 monthly laundry reimbursement
3 $200 monthly personal wellness reimbursement
3 Health, Dental, Vision insurance
