Head of Testing (AI)

Newcode.ai

$150K — $180K *
Information Technology
5 - 7 years of experience
Job Overview by Ladders

Qualifications

  • 5+ years in software testing or quality assurance
  • Experience with AI-powered products and machine learning systems
  • Strong understanding of AI evaluation methodologies
  • Skilled in designing quality processes from scratch
  • Ability to simplify technical issues into actionable insights
  • Analytical thinker with strong attention to detail
  • Cross-functional collaboration experience across teams

Responsibilities

  • Develop and execute a holistic testing strategy for AI products
  • Create comprehensive testing frameworks and methodologies
  • Implement robust processes for evaluating AI outputs
  • Build structured test plans and documentation for releases
  • Identify gaps and risks through thorough product testing
  • Collaborate with engineering to enhance product quality
  • Establish repeatable workflows for various testing phases

Benefits

  • Culture of trust and excellence promoting reliability and respect
  • Collaborative environment working on impactful legal engineering solutions
  • Opportunities for growth and ownership of your role
  • Commitment to diversity and inclusion within the team
Full Job Description
Role Overview

As Head of Testing, you will own the vision, strategy, and execution of product testing across Newcode. You will develop the frameworks, processes, and standards needed to evaluate our AI-powered products, identify opportunities for improvement, and ensure we deliver reliable, high-quality solutions to our customers.

This role requires someone who understands both software testing and AI product evaluation - someone who can think critically about how AI systems should be tested, measured, and improved.

You will work closely with Engineering and Legal Engineering to establish a world-class testing function.

Requirements
Responsibilities
  • Develop and execute Newcode's overall testing strategy across AI-powered products
  • Create testing frameworks and methodologies for evaluating product functionality, usability, accuracy, reliability, and performance
  • Design and implement processes for testing AI outputs, including quality benchmarks, evaluation criteria, and success metrics
  • Build structured test plans, test cases, and documentation to support product releases
  • Identify gaps, risks, and improvement opportunities through rigorous product testing
  • Partner with Engineering team to improve product quality and development processes
  • Establish repeatable workflows for regression testing, user acceptance testing, and release validation
  • Analyze testing results and provide actionable insights to improve product performance
  • Develop best practices for AI evaluation, including assessing accuracy, consistency, hallucination risks, and edge cases
  • Help define quality standards as Newcode scales its products and customer base
  • Build and evolve the testing function, including future team growth and processes
Qualifications
  • 5+ years of experience in software testing, quality assurance, product quality, or a related technical role
  • Proven experience testing AI-powered products, machine learning systems, or LLM-based applications
  • Strong understanding of AI evaluation methodologies and how to measure AI system performance
  • Experience designing testing frameworks and quality processes from the ground up
  • Ability to translate complex technical issues into clear insights and recommendations
  • Strong analytical and problem-solving skills with high attention to detail
  • Experience working cross-functionally with Engineering, Product, and Customer-facing teams
  • Comfortable operating in a fast-paced startup environment with changing priorities
  • Familiarity with automated testing frameworks and tooling
  • Excellent written and verbal communication skills
  • Experience building quality teams or functions within a growing organization
Nice to Have
  • Experience working with legal technology, enterprise software, or regulated industries
  • Experience evaluating LLM applications, AI agents, or generative AI workflows
  • Legal domain knowledge or experience working with legal professionals


Benefits
Why You'll Love Working with Us
  • Culture of trust and excellence: Be part of a team that values reliability, respect, and initiative, where everyone's contribution matters and diversity drives creativity.
  • Collaborative environment: Work with talented, driven peers on bleeding-edge legal engineering solutions with real-world impact.
  • Growth and ownership: Shape your role as we grow. Take ownership and drive outcomes from the start of your journey.
  • Inclusive culture of excellence and trust: We reward initiative, respect, integrity, and teamwork, and are committed to building a global, inclusive team. Your drive, curiosity, and ideas matter more than your background


Ready to shape what's next? Apply today and join us in building a smarter, more efficient world powered by AI.

Similar Jobs

More Jobs at Newcode.ai

  • Head of Testing (AI)
    $150K — $180K *
    New York, NY 10025 (New York County)
    Information Technology
    In-Person
  • Legal Engineer - New York
    $90K — $130K *
    New York, NY 10025 (New York County)
    Legal & Accounting
    In-Person

More Information Technology Jobs

Find similar Head of Testing (AI) jobs: