QA Engineer – AI and Prompt Engineering

About the Position

OpsXpress is hiring a QA Engineer specializing in manual testing and AI prompt engineering to join the core team of Emmacare, a prominent US SaaS platform focusing on Chronic Care Management and Telehealth. This role offers a full-time, remote opportunity for candidates with 4 to 5 years of QA experience and a strong background in manually testing complex software along with expertise in AI and prompt evaluation.

Key Responsibilities

• Conduct comprehensive manual testing including exploratory, functional, integration, regression, and smoke tests on both web and mobile applications.
• Perform detailed API testing with tools such as Postman, Insomnia, or Curl to verify backend operations, payload structures, status codes, and edge case handling.
• Identify complex issues like system bugs, race conditions, state management errors, and cross-platform UI inconsistencies.
• Produce clear, detailed bug reports including logs, network captures, and well-defined acceptance criteria.
• Validate complete user journeys involving multiple integrated systems.
• Analyze outputs from large language models (LLMs) and generative AI for correctness, bias, contextual relevance, hallucinations, and tone.
• Execute prompt engineering by creating, testing, and optimizing prompt templates and system instructions.
• Prepare and manage test datasets aimed at continual AI model evaluation and semantic accuracy.
• Implement standardized manual testing processes tailored to the evolving requirements of non-deterministic AI applications.
• Define and monitor quality metrics specific to AI, including relevance, coherence, response latency, and refusal accuracy.
• Engage actively in Agile development activities like sprint planning, grooming, and retrospectives as the QA representative.

Required Competencies and Experience

• At least 4 to 5 years of manual QA experience with commercial web or mobile software products.
• In-depth understanding of software architecture, client-server communication, and internet protocols.
• Expertise with API testing methodologies and tools including Postman and validation of JSON/XML payloads.
• Proficient in defect reporting and using test management platforms such as Jira, TestRail, or Qase.
• Strong grasp of Software Testing Life Cycle (STLC), exploratory testing techniques, and risk-based testing strategies.
• Documented experience assessing AI-powered features from models like OpenAI, Anthropic, Gemini, or open-source alternatives.
• Skilled in writing and tuning prompts including few-shot, chain-of-thought, and system instruction types.
• Awareness of AI-specific risks such as prompt injection, hallucinations, data poisoning, and system instruction leaks.

Preferred Skills

• Basic Python scripting for data manipulation and simple API tests.
• Knowledge of data protection and safety principles relating to AI implementations.