- Experience
- Any
- Salary
- —
- Openings
- 1
- Posted
- há 3 horas
- Work mode
- Work from home
- Eligibility
- All qualified candidates with demonstrated technical skills in programming and software quality engineering are encouraged to apply regardless of background or prior employment history.
- Resume
- Required to apply
Job description
Role Overview
We seek a Software Quality Engineer to join our client’s team remotely on a full-time basis. This position involves executing and verifying realistic coding tasks within an agentic coding framework, reviewing model trajectories, debugging code, and providing high-quality annotations. The engineer will analyze code behavior, ensure compliance with detailed evaluation rules, and deliver consistent assessments across model runs.
Key Responsibilities
- Perform coding tasks in the assigned agentic coding harness, maintaining model blindness and session independence.
- Adhere consistently to task instructions, milestones, planned interactions, and evaluation guardrails throughout evaluation runs.
- Verify outputs by reading code, executing commands, reviewing logs, and examining generated artifacts.
- Create realistic multi-step coding tasks, including defining user intents and establishing structured milestones for offline evaluations.
- Produce clear, specific, evidence-supported rationales for trajectory rankings and assessments.
Required Skills and Qualifications
- Expertise in Python, JavaScript, Java, C++, or SQL focused on code analysis and debugging.
- Experience handling realistic software challenges beyond simplistic coding or basic code review exercises.
- Capability to develop detailed task rubrics and binary evaluation criteria for software model outputs.
- Strong attention to detail to ensure precision in annotations and consistent grading.
- Comfortability with debugging tools, validating tests, and manually verifying software behaviors.
Additional Information
This opportunity allows collaboration with a global technology leader advancing powerful AI tools deployed in mission-critical applications. The role is remote, enabling work from any location, and compensation is competitive based on experience.
Equal Opportunity Statement
We are committed to inclusive hiring, welcoming all qualified applicants regardless of background or prior employment. Selection is based strictly on technical skills and qualifications.