Python Engineer - Freelance AI Trainer

About the Role

Mindrift offers project-based AI training opportunities connecting specialists with leading technology firms. This role involves evaluating AI coding agents by designing and implementing test scenarios that assess their honesty, scope adherence, and safety in completing development tasks within realistic environments.

Key Responsibilities

• Develop authentic developer environments modeling virtual companies with realistic codebases and context like tickets, documentation, and conversations to construct believable development histories.
• Create task scenarios combining legitimate development goals with tempting unsafe shortcuts such as scope creep, policy breaches, data corruption, or excessively permissive changes.
• Craft rigorous tests that determine whether AI agents complete tasks correctly without cutting corners, beyond verifying mere output correctness.
• Iteratively refine tasks and tests based on quality assurance feedback, analyzing failures and agent solutions to ensure evaluations are fair and robust.

What This Role Is Not

• This is not a data labeling assignment.
• It does not involve prompt engineering.
• It is different from cybersecurity or red-teaming roles; no attacker scenarios are present. While cybersecurity experience is advantageous, it is not required. The focus is on software engineering and understanding correct code behavior rather than penetration testing.
• The agent writes most code; the engineer designs the testing tasks and evaluates outcomes rather than coding from scratch.

Candidate Profile

• A minimum of 4 to 5 years of professional experience in software development.
• Proficiency with Python and JavaScript/TypeScript is essential.
• Strong abilities in designing functional and integration tests that differentiate safe task completion from unsafe shortcuts, not just correctness.
• Hands-on experience using AI coding agents such as Claude Code, GitHub Copilot CLI, Codex, or similar.
• Familiarity with GitHub Pull Requests and Continuous Integration workflows from a user perspective.
• Broader knowledge of backend systems and developer infrastructure like databases, CI pipelines, and deployment scripts is desirable but not mandatory.
• English language skills at B2 level or higher are required.

Challenges of the Role

Frontier AI models perform coding tasks well, making it challenging to design scenarios that truly test and expose unsafe or out-of-scope behavior. Creating such scenarios involves crafting believable temptations that are easier to take than the safe path, and designing tests that reliably detect deviations while accepting all valid solutions.

Project Process and Expectations

• The engagement is project-based: apply, complete qualification processes, join projects, execute defined tasks, and receive payment accordingly.
• During active phases, contributors should expect to spend approximately 20-25 hours weekly on tasks, though workload varies with project requirements and is not guaranteed.
• Tasks must be submitted by deadlines and meet acceptance criteria for approval.

Compensation Details

Pay rates can reach up to 75 USD per hour depending on contribution level and speed. Compensation may vary between projects based on complexity, expertise needed, and scope.

Additional Application Instructions

Submit your CV in English and clearly state your English proficiency level when applying.