5-7 years of experience in data evaluation and rubric creation
Proficiency in Python or Typescript, especially in pipeline maintenance
Hands-on experience with preference data collection
Strong written and verbal communication skills
Demonstrated ability to articulate high-quality judgments
Responsibilities
Design and iterate on evaluation rubrics for preference data
Review and label data with a focus on quality
Build and maintain data generation and evaluation pipelines
Synthesize data findings into clear recommendations
Collaborate with researchers and engineers to create scalable processes
Benefits
Fully remote work environment
Opportunity to engage with cutting-edge AI projects
Flexible working hours with a minimum overlap with Pacific Time
Full Job Description
About the Role:
We're looking for a high-taste engineer with a strong eye for quality - someone who's worked hands-on with preference data and rubric creation, and knows what separates a good example from a great one. You communicate clearly and can synthesize complex technical work the way a strong MAI dev would: concise, structured, and easy for others to act on. You're comfortable in ambiguous, fast-moving environments and default to shipping over deliberating.
What does day-to-day look like:
Design, refine, and iterate on rubrics and evaluation criteria for preference data
Review and label data with a high bar for quality, catching subtle issues others might miss
Build and maintain pipelines/infra that support data generation, collection, and evaluation workflows
Synthesize findings from data work into clear write-ups, updates, or recommendations for the team
Collaborate closely with researchers/engineers to translate qualitative judgment into scalable processes
Engagement Details:
Compensation: Market rate. Please share a specific rate expectation (rather than a range) so we can quickly assess mutual fit
Availability: 40 hours/week preferred, with a minimum 6-hour overlap with Pacific Time. If you're unable to commit to 40 hours, please share your available hours and overlap window
Employment Type: Independent contractor (does not include medical or paid leave benefits
Duration: 3 months, with an expected start date as soon as possible
Evaluation Process (approximately 30 mins):
One round of interview (30 mins of technical & cultural discussion)
Requirements
Proficiency in Python (especially building/maintaining pipelines and infra) or Typescript (experience contributing to or navigating large codebases like VSCode a plus)
Hands-on experience with preference data collection and/or rubric creation
Strong written and verbal communication - able to distill technical work into clear, actionable summaries
Demonstrated high-taste judgment - able to articulate why something is good, not just that it is.
Benefits
Work in a fully remote environment.
Opportunity to work on cutting-edge AI projects with leading LLM companies.
About Turing
Turing is a technology company that provides a platform for companies to hire remote software developers. The company's platform uses artificial intelligence to match companies with developers who have the skills and experience they need. Turing was founded in 2018 and is headquartered in San Francisco, California. The company has raised $32 million in funding to date.