- Experience
- Any
- Salary
- —
- Openings
- 1
- Posted
- 4 days ago
Job description
About Us
Gramian Consultancy is a specialized IT professional services and engineering talent solutions firm. We leverage our deep expertise in software engineering and leadership to assist companies in building effective teams by connecting them with suitable professionals.
This role is being offered on behalf of one of our clients, and we will guide you through a transparent application process.
Role Overview
We are seeking Arabic-speaking AI Content Analysts to contribute to the evaluation of a new personalization feature for a leading AI assistant platform. Your responsibilities will include crafting realistic conversational prompts based on your own experiences and assessing the AI's ability to utilize personal signals, such as past interactions or activity context, to generate relevant and accurate responses.
This position integrates creative prompt design, analytical assessment, and structured quality evaluation, making it suitable for individuals with backgrounds in AI evaluation, data annotation, content review, or analytical research.
Responsibilities
- Design and execute brief multi-turn conversations (1-5 turns) to test the AI's personalization capabilities.
- Develop prompts rooted in authentic personal scenarios to gauge contextual understanding.
- Evaluate AI-generated responses to confirm the correct application of personalization.
- Assess the quality of grounding to ensure the model avoids fabricating unsupported user claims.
- Examine integration quality, verifying that personal signals are incorporated naturally and not artificially.
- Compare two AI responses and determine which is more helpful, natural, and relevant.
- Compose clear, structured rationales for rankings, referencing specific conversational exchanges.
- Confirm debug information to ensure appropriate data sources were utilized.
- Adhere strictly to workflow protocols, including timely deletion of evaluation conversations as required.
Requirements
- Exceptional proficiency in Arabic, including reading and writing, as it is the primary language for evaluation.
- A Bachelor of Science or Bachelor of Arts degree, or equivalent experience, in a relevant discipline such as Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or a related analytical field.
- Strong analytical thinking skills and the capacity to evaluate subtle AI outputs.
- Excellent written communication abilities, with a talent for producing well-structured evaluation notes.
- Meticulous attention to detail when comparing similar AI responses.
- Capacity to work autonomously within a fully remote setting.
- A dependable desktop or laptop computer and a stable internet connection are essential.
- Readiness to use your primary personal Google account and enable personal data sources for evaluation purposes.
Additional Information
- This is a 100% remote position. Preferred countries for candidates include Egypt, Ghana, Nigeria, and Saudi Arabia.
- A commitment of 30-40 hours per week is expected. Compensation is based on approved hours logged.
- This is a contract-based role.
- The expected duration is 2 months or longer.