Data Testing Engineer

Position Overview

Join Virtusa Consulting Services Pvt Ltd as a Data Testing Engineer based in Pune, India. This role focuses on developing and implementing automated testing frameworks and strategies for data pipelines, ensuring data quality and system reliability in large-scale data environments.

Key Responsibilities

• Design and implement test automation architecture and strategies for batch and streaming data pipelines.
• Test APIs and backend services efficiently to ensure robustness.
• Engineer SQL-based assertions and reconciliation processes within the data testing framework.
• Perform schema validation and monitor schema evolution for compatibility.
• Adopt a proactive mindset to identify system weaknesses through destructive testing methods, including schema drift and backward incompatibility.
• Handle scenarios such as late-arriving data and partial failures.
• Execute validation of ETL and streaming data at scale using row-level, aggregate, and hash-based reconciliation, as well as incremental and backfill validation.
• Integrate data quality and observability tools like Great Expectations and Soda to monitor accuracy, completeness, uniqueness, and timeliness of data.
• Enforce Continuous Integration/Continuous Deployment (CI/CD) and DataOps practices, including pre-merge gates and release blockers with selective and parallel test execution.
• Manage test data through synthetic data generation, masking, and anonymization techniques.
• Conduct performance and reliability tests establishing benchmarks for pipelines and queries, including concurrency and stress testing.
• Validate security and compliance by monitoring Personally Identifiable Information (PII) or Protected Health Information (PHI) exposures, encryption, and access controls.
• Collaborate cross-functionally with data engineers, platform teams, and architects to embed quality at early stages and challenge suboptimal design decisions affecting reliability.
• Analyze incidents and production data failures to reduce flaky tests and minimize false alarms.

Required Skills and Expertise

• Strong programming capabilities and experience in framework development.
• Profound knowledge of the data engineering ecosystem.
• Expertise in data quality assurance and observability practices.
• Familiarity with streaming data systems.
• Experience with CI/CD and DevOps methodologies.
• Understanding of cloud infrastructure and related technologies.

Preferred Skills

• Hands-on experience with API contract testing tools such as PACT.
• Experience in basic UI automation for end-to-end validation.
• Exposure to data mesh concepts or data product architectures.
• At least 5 years of experience in automation or quality engineering roles.
• Minimum 3 years spent building frameworks for scalable data platforms.

Additional Information

This position does not require graduation for eligibility. The salary range offered is between 1,000,000 and 1,800,000 INR per year. The role is based in Pune and requires onsite work.