Data/Analytics Lead

About the Role

We are recruiting for an innovative technology company focused on developing sovereign, secure, and scalable digital platforms spanning AI, Web3.0, cloud, and health technology.

As the Data/Analytics Lead, you will be responsible for architecting and executing a cutting-edge analytics platform based purely on open-source technologies. This role includes creating scalable data pipelines, building a lake house architecture, and establishing a comprehensive business intelligence infrastructure from the ground up. You will lead a team of 4-6 data engineers.

Key Responsibilities

• Guide and manage a team of 4-6 data engineers.
• Define the architecture and strategic roadmap for the analytics platform.
• Build the data platform from scratch utilizing open-source solutions.
• Set up data governance policies and quality assurance frameworks.
• Design and implement an end-to-end analytics platform using an open-source technology stack.
• Construct a data lake house leveraging Cloudian object storage.
• Architect OLAP workloads with ClickHouse to achieve sub-second query performance.
• Design multi-source data ingestion methods incorporating APIs, databases, and SaaS platforms.
• Develop and maintain real-time and batch data pipelines.
• Create ETL pipelines with Airbyte supporting change data capture, connectors, and validation.
• Manage workflow orchestration using Apache Airflow with DAGs, monitoring, and error handling.
• Develop incremental and full-load data ingestion strategies and utilize Kafka for data streaming.
• Enhance ClickHouse performance via materialized views and distributed tables.
• Design dimensional data models such as star and snowflake schemas.
• Build semantic layers to maintain consistent metrics.
• Deploy Apache Superset for self-service analytics and optionally integrate Power BI.
• Implement Redis caching to boost performance.
• Implement data quality frameworks including Great Expectations and Soda.
• Automate data reconciliation processes.
• Establish data lineage, cataloging, and ensure compliance with regulations like GDPR.
• Create Python automation tools, develop custom connectors, and implement CI/CD for data pipelines.
• Continuously optimize system costs and performance.

Candidate Eligibility

Applicants must have at least 6 years of professional experience and hold a degree in Computer Science Engineering or related fields.

Compensation and Benefits

The annual salary range offered for this position is ₹2,500,000 to ₹3,200,000. Selected candidates will receive a formal job offer.

Additional Information

Immediate joining is expected.
Location: Noida, Uttar Pradesh, India (onsite)