Senior Data Engineer

Role Overview

We are seeking a Senior Data Engineer with extensive experience in data engineering to join our team in Hyderabad, India. The ideal candidate will have 10 to 15 years of expertise in developing scalable data ingestion frameworks and working extensively with AWS SageMaker Unified Studio and related AWS services.

Key Responsibilities

• Utilize AWS SageMaker Unified Studio components such as Discover, Build, and Govern for development activities.
• Develop within SageMaker IDE and applications including Jupyterlab, Spaces, and Partner AI Apps.
• Perform data analysis and integration by leveraging Query Editor, Visual ETL Jobs, and data processing workflows.
• Orchestrate workflows and machine learning pipelines efficiently.
• Setup projects and maintain data governance protocols within SageMaker Unified Studio.
• Create advanced data ingestion and processing frameworks for both batch and real-time data streams.
• Design reusable and adaptable data ingestion frameworks compatible with various source types including databases, APIs, message queues, and file systems.
• Build low-latency real-time data pipelines with strict sub-second latency using technologies such as AWS Kinesis and Apache Kafka.
• Implement batch ingestion processes using tools like AWS Glue and Apache Spark for large-scale data processing.
• Develop efficient data transformation logic for streaming and batch workflows.
• Write optimized Python scripts to build scalable ingestion/export frameworks, integrate APIs, and utilize AWS SDK (Boto3).
• Apply performance optimization techniques for data-heavy Python applications.
• Possess familiarity with additional programming languages like Scala and Spark for high-performance streaming applications.
• Maintain comprehensive knowledge of AWS services including Lambda, S3, DynamoDB, CloudWatch, SQS, SNS, and API Gateway.
• Ensure secure infrastructure management through AWS networking concepts such as VPC, security groups, private endpoints, and implement fine-grained IAM access controls.

Required Qualifications and Eligibility

• Bachelor's degree in Engineering (B.Tech/B.E.) in any specialization.
• Applicants with demonstrated expertise in data engineering and AWS service integrations are preferred.

Work Location and Other Details

This is an onsite full-time position based in Hyderabad, India. No salary information or number of openings has been disclosed.