- Esperienza
- 5+ anni
- Stipendio
- USD 50 / hour
- Aperture
- 1
- Pubblicato
- 3 settimane fa
- Modalità di lavoro
- In ufficio
- Requisiti di ammissibilità
- Professionals with at least 5 years of software development experience and proficiency in Python, JavaScript/TypeScript, Docker, Postgres, Kafka, and Redis. Candidates must also have experience in writing tests and possess B2+ English proficiency.
- Riprendere
- È necessario candidarsi
Descrizione del lavoro
About Mindrift
Mindrift specializes in connecting skilled professionals with project-based opportunities in artificial intelligence, focusing on the testing, evaluation, and enhancement of AI systems for prominent technology firms. Participation is structured around specific projects rather than permanent employment.
Project Overview: AI Coding Agent Evaluation
This project involves the creation of a comprehensive dataset designed to assess the capabilities of AI coding agents. The goal is to determine how effectively these agents can handle authentic developer tasks.
Key Responsibilities
- Construct realistic developer environments, simulating a virtual company with a complete codebase, necessary infrastructure, and contextual information (including tickets, documentation, and communications) to establish a credible development history.
- Develop challenging tasks and define precise evaluation criteria within these simulated environments. This includes crafting effective prompts and establishing clear definitions of what constitutes a