Data Science Expert - Remote

Role Overview

Join a partnership with a prominent AI research institution as an experienced data scientist contributing to a project that assesses AI systems' proficiency in performing practical data science tasks. Instead of generating deliverables yourself, your focus will be on establishing what constitutes outstanding work by crafting specific grading standards and rigorously evaluating completed work samples with well-founded written explanations.

Key Responsibilities

• Develop clear, task-oriented grading criteria for diverse real-world data science outputs such as analytics, predictive models, dashboards, experimental reports, and strategic recommendations.
• Evaluate AI-produced and human-generated work samples using these criteria, providing comprehensive written reasoning for each assigned score.
• Maintain consistency and evidence-based judgment ensuring reproducibility and defensibility of scoring outcomes.
• Incorporate structured feedback from senior reviewers and promptly refine your evaluations accordingly.

Ideal Qualifications

• At least five years of professional experience in data science within the industry.
• Experience specifically in business operations, product development, or growth-oriented data science roles at leading tech companies.
• Strong expertise in experimental design and A/B testing, metric creation, SQL and Python analytical skills, and effective communication of insights to executive leadership.
• Excellent written communication abilities.
• Highly detail-focused, consistent in judgment, and open to peer review and calibration.
• Prior involvement with AI training, evaluation activities, or human-data interaction projects is highly advantageous.

Application Process

• Submit your resume or relevant technical credentials to initiate.
• Qualified candidates may undergo a brief technical evaluation or provide supplementary information.

Contract and Payment Conditions

• Engagement will be as an independent contractor.
• This position is fully remote with flexible scheduling.
• Project durations may vary, including possible extension, shortening, or early termination based on requirements and performance.
• Payments are disbursed weekly via Stripe or Wise in proportion to services performed.

We welcome applications from all qualified individuals regardless of legally protected characteristics and are ready to offer reasonable accommodations when requested.