AI Evaluation Specialist

Position Overview

We are seeking an AI Evaluation Specialist to join our team on a contract basis. This remote role offers an hourly pay of $20 to $35, with a flexible commitment of 10 to 40 hours per week.

Key Responsibilities

• Create and execute independent evaluation assignments, including crafting prompts, preparing necessary supporting documents, and developing comprehensive grading rubrics to effectively measure AI capabilities in practical computer-guided tasks.
• Establish clear, precise written standards for judging successful and unsuccessful task outcomes across a variety of administrative and workflow contexts.
• Carefully observe and record AI behaviors, generating accurate and concise English reports and summaries.
• Continuously improve evaluation tasks and scoring rubrics through iterative feedback and close collaboration with the team to maintain strong benchmarking approaches.
• Work alongside the client's team to share findings and contribute to ongoing enhancement of AI evaluation methods.

Candidate Requirements

• At least some relevant professional experience involving written accuracy and analytical thinking, such as roles like paralegal, executive assistant, junior analyst, librarian, document archivist, research assistant, technical writer, or quality assurance analyst.
• Native or fluent English proficiency with proven capability to produce clear, concise, and unambiguous written observations.
• Demonstrated expertise in creating or applying rubric-based assessment systems, grading criteria, or structured scoring frameworks.
• Exceptional attention to detail with the ability to detect subtle patterns or inconsistencies that others may overlook.
• Strong written and verbal communication skills to clearly document nuanced observations and provide effective feedback.

Application Process

• Submit your application through the designated platform.
• Monitor your email for further instructions.
• Complete a resume evaluation and interview as part of the selection procedure.