LLM Evaluation Specialist – Analytical Reviewer

Role Overview

This is a remote hourly contract position paying $70 per hour, with an expected commitment of 10 to 40 hours per week. The specialist will focus on analyzing and evaluating AI-generated responses to support AI research efforts.

Key Responsibilities

• Assess AI-generated outputs for quality and accuracy.
• Deliver well-structured, written feedback on evaluated responses.
• Utilize specific and detailed evaluation guidelines to guide assessments.
• Contribute to the advancement of AI research via reliable and consistent evaluations.

Required Qualifications

• Excellent analytical thinking paired with strong written communication skills.
• Capability to work autonomously while strictly adhering to comprehensive task instructions.
• High-level critical reading ability capable of detecting subtle meanings, implicit messages, and reasoning gaps.
• Skill in crafting clear, precise, and well-supported written explanations, extending beyond superficial comments.
• Demonstrated consistency and integrity in providing judgement, including readiness to give critical feedback when necessary.
• Meticulous attention to detail with flawless implementation of structured evaluation criteria.
• Comfortable performing tasks without the assistance of AI writing tools.

Application Process

• Submit a resume upload.
• Participate in a 15-minute interview.
• Complete a submission form.