⚠️
This job is no longer accepting applications.
The application deadline has passed. Browse open jobs →
- Experience
- Any
- Salary
- USD 20 – USD 35 / hour
- Openings
- 1
- Posted
- 1 week ago
- Work mode
- Work from home
- Eligibility
- Professionals with a background in detail-oriented, writing-heavy, and structured-thinking roles, and who can work fluently in English, are suitable for this opportunity.
- Resume
- Required to apply
Job description
Role Overview
This contract role focuses on creating and refining evaluation tasks that measure how well AI systems handle practical computer-based work. The assignment is fully remote and is expected to require 10 to 40 hours per week. Pay is set at $20 to $35 per hour.
What You'll Do
- Build self-contained assessment tasks from the ground up, including prompts, reference materials, and detailed grading guides for judging AI performance in realistic workflow settings.
- Write clear success and failure standards so task outcomes can be judged consistently across a range of administrative and process-driven scenarios.
- Carefully track AI agent actions and turn those observations into concise, accurate reports written in strong English.
- Improve tasks and scoring guides over time by incorporating feedback and working closely with the team to strengthen evaluation quality.
- Partner with the client’s team to exchange findings and support ongoing improvements in AI assessment methods.
What We're Looking For
- Relevant background in work that depends on precise writing and structured reasoning, such as paralegal work, executive assistance, junior analysis, library work, document archiving, research assistance, technical writing, or QA analysis.
- Strong English writing ability, with a proven capacity to produce notes and observations that are clear, exact, and not open to misinterpretation.
- Hands-on experience with rubric-driven evaluation, scoring against defined criteria, or creating structured assessment frameworks.
- Excellent attention to detail and the ability to spot small inconsistencies or patterns that may be easy to overlook.
- Strong communication skills in both written and spoken form, especially for documenting nuanced feedback and observations.
Application Steps
Applicants are expected to complete a simple application flow, then monitor email for follow-up instructions and proceed through resume review and an interview stage.
Additional Information
This is a contract position based remotely from the United States. The role is structured for 10 to 40 hours each week. Compensation is hourly at $20 to $35, and the position is designed for professionals who can contribute careful evaluation work with a high level of accuracy.