- 경험
- 6년 이상
- 샐러리
- INR 2,500,000 – INR 3,200,000 / year
- 채용 공고
- 1
- 게시됨
- 3시간전
- 작업 모드
- 사무실에서
- 교육
- 컴퓨터 과학 공학
- 적임
- Candidates must have at least 6 years of relevant professional experience and be graduates in Computer Science Engineering.
- 재개하다
- 신청 시 필수 사항
당신이 일하게 될 곳
직무 설명
About the Role
We are recruiting for an innovative technology company focused on developing sovereign, secure, and scalable digital platforms spanning AI, Web3.0, cloud, and health technology.
As the Data/Analytics Lead, you will be responsible for architecting and executing a cutting-edge analytics platform based purely on open-source technologies. This role includes creating scalable data pipelines, building a lake house architecture, and establishing a comprehensive business intelligence infrastructure from the ground up. You will lead a team of 4-6 data engineers.
Key Responsibilities
- Guide and manage a team of 4-6 data engineers.
- Define the architecture and strategic roadmap for the analytics platform.
- Build the data platform from scratch utilizing open-source solutions.
- Set up data governance policies and quality assurance frameworks.
- Design and implement an end-to-end analytics platform using an open-source technology stack.
- Construct a data lake house leveraging Cloudian object storage.
- Architect OLAP workloads with ClickHouse to achieve sub-second query performance.
- Design multi-source data ingestion methods incorporating APIs, databases, and SaaS platforms.
- Develop and maintain real-time and batch data pipelines.
- Create ETL pipelines with Airbyte supporting change data capture, connectors, and validation.
- Manage workflow orchestration using Apache Airflow with DAGs, monitoring, and error handling.
- Develop incremental and full-load data ingestion strategies and utilize Kafka for data streaming.
- Enhance ClickHouse performance via materialized views and distributed tables.
- Design dimensional data models such as star and snowflake schemas.
- Build semantic layers to maintain consistent metrics.
- Deploy Apache Superset for self-service analytics and optionally integrate Power BI.
- Implement Redis caching to boost performance.
- Implement data quality frameworks including Great Expectations and Soda.
- Automate data reconciliation processes.
- Establish data lineage, cataloging, and ensure compliance with regulations like GDPR.
- Create Python automation tools, develop custom connectors, and implement CI/CD for data pipelines.
- Continuously optimize system costs and performance.
Candidate Eligibility
Applicants must have at least 6 years of professional experience and hold a degree in Computer Science Engineering or related fields.
Compensation and Benefits
The annual salary range offered for this position is ₹2,500,000 to ₹3,200,000. Selected candidates will receive a formal job offer.
Additional Information
Immediate joining is expected.
Location: Noida, Uttar Pradesh, India (onsite)