Head of Testing (AI)

About Newcode.ai

Newcode.ai is revolutionizing how law firms and legal professionals utilize artificial intelligence to create tangible impacts. As a member of our dynamic, rapidly expanding team based in New York, you'll collaborate directly with pioneering founders at the forefront of AI and legal innovation, influencing both product development and the future landscape of legal work.

Our startup thrives on adaptability and growth, with evolving processes and shifting priorities. We seek individuals who are comfortable navigating ambiguity, take initiative independently, and contribute proactively to building the company alongside us.

Role Overview

As the Head of Testing, you will lead and own the comprehensive vision, strategy, and operational execution of testing for our AI-driven products. You will formulate testing frameworks, methodologies, and standards to thoroughly assess functionality, usability, accuracy, reliability, and performance, ensuring that Newcode delivers dependable, high-quality AI solutions to clients.

Key Responsibilities

• Develop and implement a cohesive testing strategy across all AI-powered offerings.
• Create robust testing methodologies and frameworks that evaluate product functionality, user experience, accuracy, reliability, and system performance.
• Design processes to test AI output quality, including benchmarks, evaluation criteria, and success metrics.
• Build detailed test plans, design test cases, and maintain documentation to facilitate smooth product releases.
• Identify potential risks, gaps, and areas for enhancement through rigorous evaluation.
• Collaborate closely with Engineering teams to enhance product quality and development workflows.
• Establish systematic approaches for regression, user acceptance, and release validation testing.
• Interpret testing outcomes and deliver actionable recommendations to improve performance.
• Develop best practices for AI assessment focusing on accuracy, consistency, hallucination risk, and edge case scenarios.
• Define and uphold quality standards to support product scaling and expanding clientele.
• Lead and expand the testing function, including planning for team growth and process improvements.

Required Qualifications

• Minimum of 5 years experience in software testing, quality assurance, or product quality assurance.
• Demonstrated expertise in testing AI products, machine learning models, or applications based on large language models.
• Deep knowledge of AI evaluation techniques and performance measurement methodologies.
• Experience developing testing frameworks and quality processes from inception.
• Ability to transform complex technical challenges into clear insights and practical recommendations.
• Strong analytical and detail-oriented problem-solving skills.
• Proven ability to work across Engineering, Product, and customer-facing teams effectively.
• Adaptability to a fast-evolving startup environment with shifting priorities.
• Familiarity with automated testing tools and frameworks.
• Excellent communication skills, both written and verbal.
• Experience building and leading quality assurance teams within growth-stage organizations.

Preferred (Nice to Have)

• Background in legal technology, enterprise software, or compliance-heavy industries.
• Experience evaluating applications involving large language models, AI agents, or generative AI workflows.
• Legal domain expertise or history of working with legal professionals.

Why Join Us?

• Culturally rich environment valuing trust, excellence, respect, and initiative with an inclusive and diverse team.
• Collaborative workplace with highly driven professionals creating cutting-edge legal engineering solutions.
• Opportunities for personal growth and empowerment to own your role and influence company outcomes from the outset.
• Commitment to integrity, teamwork, and fostering a global, inclusive culture where your ideas are valued above all.

Additional Information

This role is onsite in New York City as part of a fast-moving startup focused on revolutionary AI-powered legal tools. Candidates should be ready to thrive in a dynamic setting where processes evolve rapidly and ownership is key.