- 经验
- 4–5 yrs
- 薪水
- USD 75 – USD 75 / hour
- 职位空缺
- 1
- 发布
- 2小时前
- 工作模式
- 在办公室
- 恢复
- 需要申请
职位描述
Overview
Mindrift facilitates connections between expert professionals and project-based AI initiatives for renowned technology firms. This role involves evaluating and enhancing AI systems through temporary, project-focused engagements rather than permanent employment.
Responsibilities
- Create authentic developer settings, including a simulated company environment with codebases, infrastructure, and contextual materials like tickets, documentation, and discussions, establishing a consistent development history.
- Devise tasks that combine legitimate development objectives with enticing unsafe shortcuts, such as policy violations, scope extensions, data inconsistencies, or overly permissive alterations.
- Develop tests that ascertain if the AI agent completes assignments correctly and safely, identifying when shortcuts or corner-cutting occur rather than solely verifying final outputs.
- Continuously refine tasks and assessments by reviewing agent solutions, analyzing failure cases, and incorporating QA feedback to ensure fair and reliable evaluations.
What This Role is Not
- This position does not involve data labeling or prompt engineering.
- It excludes cybersecurity or adversarial testing—there is no adversary in the scenario. While cybersecurity knowledge is beneficial, the role focuses on understanding appropriate code behavior rather than penetration testing.
- Developing code from scratch is not required since the AI agent generates most code; you are responsible for designing the scenarios and assessing results.
Required Qualifications
- A minimum of 4-5 years of experience in software development.
- Proficiency with Python and JavaScript/TypeScript as primary technologies.
- Expertise in crafting functional and integration tests that distinguish between safe and unsafe task completions, not just correct versus incorrect outcomes.
- Practical experience working with AI coding agents such as Claude Code, GitHub Copilot CLI, Codex, or equivalents.
- Familiarity with GitHub pull requests and continuous integration workflows from a user perspective.
- Broader experience with backend systems and infrastructure components is advantageous, as tasks simulate comprehensive repositories including databases, CI pipelines, and deployment scripts.
- English language proficiency at B2 level or higher.
Challenges
Leading AI models already perform well in coding tasks, making it challenging to create assignments that truly test these systems. The key difficulty lies in designing scenarios where the unsafe or out-of-scope option appears easiest, and developing tests that reliably detect such behavior while accepting all valid solutions.
Engagement Details
- The application process involves submission, qualification, project assignment, task completion, and compensation.
- Active project phases may entail approximately 20-25 hours weekly, though workload is estimated and not guaranteed; tasks must meet deadlines and quality standards for acceptance.
Compensation
This role offers remuneration up to 75 USD per hour, varying by contribution level and speed. Compensation may differ across projects depending on complexity and expertise required.
Additional Instructions
Applicants should submit their CV in English and clearly indicate their English proficiency level.