Job description

About the Role

We are seeking a highly detail-oriented AI Quality Analyst focused on personalization to join our remote team on a short-term contract basis. This role involves designing and evaluating multi-turn conversational AI prompts that rely on personal context, ensuring the outputs are accurate, relevant, and helpful.

Key Responsibilities

Create multi-turn conversational prompts tailored to individual personal data contexts.
Critically evaluate AI-generated personalized responses for relevance, accuracy, and grounding.
Analyze the appropriate use or misuse of personal information within model outputs.
Conduct side-by-side comparisons and rank multiple AI responses to determine best answers.
Identify errors related to grounding, flawed inferences, or forced personalization in AI behaviors.
Document clear and structured rationales citing exact conversation turns as evidence.
Extract and confirm debug data and sources the AI model utilized during conversations.
Ensure strict data hygiene by securely deleting conversations after evaluation.

Candidate Requirements

Proficiency in English, both reading and writing.
Demonstrated experience in data annotation, AI quality evaluation, content moderation, or similar analytical roles.
Strong analytical skills coupled with keen attention to detail.
Capability to assess nuanced and subtle AI-generated responses accurately.
Comfortable using a personal Google account with enabled data sources for evaluation purposes.
Bachelor's degree in a relevant analytical discipline or equivalent professional experience.
Effective written communication skills with ability to provide structured, constructive feedback.
Self-driven, able to work independently within a remote environment.
Access to a reliable desktop or laptop computer and stable internet connectivity.

Engagement Details

Contract duration: One month.
Work hours: Approximately 20 hours per week.
Compensation: $11.5 per hour.
Start date: Immediate availability required.

Application Process

Complete and submit the application form.
Fill out the ICF (Independent Contractor Form).
Undergo the assessment phase.

AI Quality Analyst (English) - Remote Contract