Language Model Analyst - English & Kannada (Remote, Part-time)

About the Role

Mercor is a platform linking top creative and technical talent with foremost AI research entities. Based in San Francisco, the company is backed by notable investors including Benchmark, General Catalyst, and prominent individual investors.

This role is a contract position working remotely, offering hourly pay between $15 and $20 as a Language Model Analyst focusing on English and Kannada languages.

Key Responsibilities

• Verify facts by consulting reliable public sources and auxiliary tools.
• Create detailed human evaluation data by identifying strengths, weaknesses, and factual errors in model outputs.
• Evaluate responses for reasoning quality, clarity, tone, and completeness.
• Ensure the AI model’s replies conform to conversational norms and organizational guidelines.

Required Qualifications

• Bachelor’s degree.
• Native proficiency in Kannada language.
• Substantial experience working with large language models (LLMs).
• Strong English writing capabilities.
• Exceptional attention to detail.
• Background or prior work involving structured, analytical thought processes.

Preferred Qualifications

• Prior experience with Reinforcement Learning from Human Feedback (RLHF), model evaluation, or annotation tasks.
• Experience in crafting or editing well-written content.
• Skill in comparing multiple outputs with nuanced qualitative assessments.

Application Process

• Submit a resume.
• Complete an AI-based interview tailored to your resume.
• Fill out the accompanying application form.

Additional Information

• The application review occurs daily; applicants are encouraged to complete all steps promptly.
• For details on the interview and platform, refer to internal resources.
• Support is available via email for any application-related queries.