About the RoleThis is the top hiring priority for a ~15-person engineering team working on infrastructure for building, training, and evaluating AI models in reinforcement learning environments. As Research Engineer, QC Automation, you'll own the systems that scale quality control for training data generated across the platform - combining deep human judgment with rigorous engineering to ensure data quality keeps pace with demand.
What You'll Do- Automate quality control for RL training data produced by external data vendors using the platform.
- Build QC systems grounded in true understanding and human judgment, without heavy reliance on LLMs.
- Define and enforce quality standards for post-training datasets.
- Design experiments and metrics to grade agent outputs and evaluate task trajectories.
- Partner with data vendors to debug quality issues, diagnose agent failure modes, and improve their data generation processes.
- Translate QC findings into auditing systems, including sampling strategies and rule-based or model-assisted validation pipelines.
- Feed QC learnings back into infrastructure tooling and vendor-facing portals to reduce anomalies and edge cases over time.
What We're Looking For- 2-4 years of experience in research engineering or a similar role with a focus on QC automation.
- Proficiency in Python, Docker, and Linux environments.
- Proven ability to build scalable data validation pipelines and automated QA/QC systems end-to-end, without a fully prescribed roadmap.
- Experience working on benchmarks and evaluations for RL training data - defining realistic tasks, reliable rubrics, and useful agent trajectories.
- Strong sense of what "good data" looks like and how to measure it rigorously.
- Experience creating QC systems driven by human understanding rather than LLM reliance.
- Solid statistics knowledge and comfort designing metrics, experiments, and QA/QC processes.
- Clear written and verbal communication skills for cross-timezone collaboration.
- Genuine curiosity, intellectual range, and the ability to thrive in fast-paced, unstructured environments.
Compensation & BenefitsSalary range:
$150,000 - $250,000 USD annually. Visa sponsorship is available.
LocationOn-site in
San Francisco, CA for US-based candidates, or
Singapore for Southeast Asia-based candidates. Fully remote independent contractor arrangements are also considered for candidates based elsewhere, particularly in Europe.
