Descrição da vaga

Role overview

This is a short-term remote contract for an LLM AI Quality Analyst focused on personalization. The assignment runs for one month, requires 20 hours per week, and begins immediately. Compensation is set at $11.5 per hour.

What you'll do

Create multi-turn dialogue prompts that use personal context.
Review personalized AI outputs for usefulness, relevance, and whether they are grounded in the provided context.
Check whether the model uses personal information correctly or incorrectly.
Compare two responses side by side and rank them based on quality.
Spot grounding mistakes, weak assumptions, and overdone personalization.
Prepare concise, well-organized explanations that refer to specific turns in the conversation.
Pull out and validate model debug details and evidence of data-source usage.
Keep evaluation data clean by removing conversation records after review.

Requirements

Strong command of English, both written and reading.
Background in data annotation, AI evaluation, content moderation, or a similar field.
Sharp analytical ability and a strong eye for detail.
Comfort evaluating subtle, unclear, or ambiguous AI responses.
Willingness to use a primary personal Google account with data sources enabled.
BS/BA degree, or equivalent relevant analytical experience.
Clear written communication and the ability to give structured feedback.
Ability to work independently in a remote environment.
Access to a dependable desktop or laptop with stable internet.

Application process

Submit the application form.
Complete the ICF.
Finish the assessment.

Engagement details

This is a short-term contract role based remotely, with a one-month engagement and an immediate start. The work commitment is 20 hours per week.

LLM AI Quality Analyst (Personalization) - English