Data Science Expert - Remote

Role Overview

Join a premier AI research organization as an experienced data scientist to assess the performance of AI systems on authentic data science tasks. Instead of producing outputs, you will establish benchmarks for excellence by creating precise grading standards and evaluating submitted work samples with thorough, reasoned explanations.

Key Responsibilities

• Develop detailed, task-specific grading rubrics for diverse data science outputs such as analyses, predictive models, dashboards, experimental reports, and recommendations.
• Evaluate both AI-generated and human-submitted work samples against these criteria, providing comprehensive written justifications for each assigned score.
• Maintain consistent, evidence-based scoring to ensure reproducibility and defensibility of the evaluations.
• Incorporate structured feedback from senior reviewers and rapidly refine your evaluations accordingly.

Ideal Qualifications

• More than five years of professional experience practicing data science within industry environments.
• Expertise in business operations, product, or growth analytics, preferably at leading technology firms.
• Strong proficiency in experimental design including A/B testing, defining metrics, performing SQL and Python data manipulations, and effectively communicating analytical outcomes to executive leadership.
• Superior written communication skills.
• Highly meticulous and consistent in judgment, comfortable with peer review and calibration processes.
• Experience with AI training, evaluation, or human-data interaction projects is an advantageous plus.

Application Process

Submit a resume or documentation detailing your technical background to initiate consideration. Selected candidates may be required to complete a concise technical assessment or provide supplementary information.

Additional Information

• Equal opportunity employer: applications are evaluated without regard to legally protected characteristics, and reasonable accommodations are available upon request.
• This is a contract-based position engaged as an independent contractor.
• Fully remote work with flexible scheduling.
• Contract duration may vary depending on project requirements and individual performance.
• Remuneration is paid weekly via Stripe or Wise based on completed work.