AI Quality Analyst (English) - Remote Contract

About the Role

We are seeking a highly detail-oriented AI Quality Analyst focused on personalization to join our remote team on a short-term contract basis. This role involves designing and evaluating multi-turn conversational AI prompts that rely on personal context, ensuring the outputs are accurate, relevant, and helpful.

Key Responsibilities

• Create multi-turn conversational prompts tailored to individual personal data contexts.
• Critically evaluate AI-generated personalized responses for relevance, accuracy, and grounding.
• Analyze the appropriate use or misuse of personal information within model outputs.
• Conduct side-by-side comparisons and rank multiple AI responses to determine best answers.
• Identify errors related to grounding, flawed inferences, or forced personalization in AI behaviors.
• Document clear and structured rationales citing exact conversation turns as evidence.
• Extract and confirm debug data and sources the AI model utilized during conversations.
• Ensure strict data hygiene by securely deleting conversations after evaluation.

Candidate Requirements

• Proficiency in English, both reading and writing.
• Demonstrated experience in data annotation, AI quality evaluation, content moderation, or similar analytical roles.
• Strong analytical skills coupled with keen attention to detail.
• Capability to assess nuanced and subtle AI-generated responses accurately.
• Comfortable using a personal Google account with enabled data sources for evaluation purposes.
• Bachelor's degree in a relevant analytical discipline or equivalent professional experience.
• Effective written communication skills with ability to provide structured, constructive feedback.
• Self-driven, able to work independently within a remote environment.
• Access to a reliable desktop or laptop computer and stable internet connectivity.

Engagement Details

• Contract duration: One month.
• Work hours: Approximately 20 hours per week.
• Compensation: $11.5 per hour.
• Start date: Immediate availability required.

Application Process

• Complete and submit the application form.
• Fill out the ICF (Independent Contractor Form).
• Undergo the assessment phase.