Researcher, Benchmark Reviews

Role overview

This opportunity is for a Researcher focused on benchmark reviews, working on behalf of a partner company that handles the application process and all subsequent hiring steps. The role is based in the United States and centers on creating detailed, public research that evaluates AI benchmarks used to compare and understand machine learning systems.

You will study newly released benchmarks in depth, examine how they are built, and assess what their results really say about model performance. The work combines research, writing, and data analysis, and it requires sound independent judgment, comfort with complex experimental setups, and the ability to interpret dense evaluation frameworks with precision. The output will be maintained as an evolving research resource for both technical and non-technical audiences.

This position is well suited to someone who enjoys questioning assumptions, working through complicated benchmark structures, and contributing to clearer understanding and transparency in AI evaluation.

Accountabilities

• Regularly assess new AI benchmarks by reviewing their design, structure, methodology, and the meaning of their reported results.
• Write polished, publication-ready research pieces that explain benchmark strengths, weaknesses, and interpretation in a way that is accessible to a wide readership.
• Work through benchmark datasets and individual tasks in detail, using coding tools or agents as support while still directing the analysis yourself.
• Keep published reports current as benchmarks change and as new models alter the performance landscape.
• Suggest missing or improved benchmark ideas by spotting gaps in the current evaluation ecosystem and proposing new directions.
• Translate technically complex evaluation systems into clear, understandable findings for analysis and communication.

Requirements

• Excellent writing skills with the ability to create structured, high-quality research content with minimal editorial support.
• Practical knowledge of AI and machine learning benchmarks, including how they are designed, where they excel, and where they fall short.
• Strong critical reasoning skills for spotting weak research design, methodological issues, and overextended benchmark claims.
• Comfort working with large, technical datasets and breaking complicated evaluation setups into usable insights.
• Ability to use coding tools or AI agents to support analysis while maintaining full ownership of interpretation and judgment.
• Genuine interest in AI research, evaluation practices, and the broader impact of benchmarking on model development.
• Previous experience writing about AI or working in evaluation is advantageous, but not essential.

Perks and benefits

• Annual compensation in the range of USD 100,000 to USD 200,000, depending on experience and location.
• Remote-first setup with flexible working hours.
• Global benefits package that may include health coverage, life insurance, and pension support where applicable.
• Paid time off that includes protected annual leave, flexible sick leave, and paid parental leave.
• Allowance for equipment, software, learning resources, and approved AI tools.
• Opportunities to travel for retreats and conferences.
• Optional access to office space with amenities such as meals and fitness facilities.
• Inclusive working culture that supports collaboration across time zones.

Additional information

This role is handled through a partner hiring process. Applications are reviewed using an AI-assisted matching system, and the shortlisted candidates are shared with the hiring company for interviews, assessments, and final decisions. Personal data may be processed for recruitment purposes under applicable data protection laws, including GDPR, with rights such as access, correction, deletion, and objection available to applicants. AI tools may support parts of the recruitment workflow, but human judgment remains responsible for final hiring decisions.