Senior Data Engineer - AWS Glue, SQL, Python

About the Role

We are seeking a seasoned Senior Data Engineer specializing in AWS technologies, specifically AWS Glue, Python, and SQL, to join our team in Hyderabad. This full-time position calls for a professional with 3 to 6 years of experience in data engineering to architect, build, and sustain scalable data solutions that support data-driven decisions and business growth.

Key Responsibilities

• Design and manage ETL and data transformation pipelines primarily using AWS Glue.
• Create and optimize scalable data processing frameworks to handle large datasets efficiently.
• Execute complex SQL queries for data extraction, transformation, and analysis purposes.
• Develop and maintain Python scripts to automate and enhance data workflows.
• Implement continuous integration and deployment processes through robust DevOps methodologies.
• Monitor and troubleshoot current data pipelines, enhancing performance and reliability.
• Collaborate with cross-functional teams to gather data requirements and deliver effective data solutions.
• Uphold data quality and enforce governance and security policies rigorously.
• Support initiatives related to data warehousing and data lakes within AWS cloud infrastructure.
• Keep abreast of the latest trends and best practices in cloud-based data engineering.

Essential Qualifications and Skills

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a closely related discipline.
• Three to six years of practical experience in data engineering roles.
• Demonstrated expertise in AWS Glue and ETL pipeline development.
• Strong command of Python programming and SQL (MS SQL Server preferred).
• Proven experience with DevOps tools and automated deployment workflows (CI/CD).
• Good understanding of data warehousing, data lakes, and data modeling principles.
• Excellent problem-solving abilities coupled with effective communication skills.
• Certifications such as AWS Certified Data Engineer, AWS Certified Solutions Architect, AWS Certified Developer Associate, and relevant Python or DevOps certificates are a plus.

Preferred Skills

• Familiarity with AWS Lambda, Amazon S3, and AWS Redshift services.
• Experience with PySpark, Apache Airflow, and data lake architecture.
• Knowledge of infrastructure as code tools like Terraform or CloudFormation.
• Proficiency with version control and CI/CD tools like Git and Jenkins.
• Exposure to big data ecosystems and technologies.