Enhancements, new development, defect resolution, and production support of ETL development using AWS native services
Integration of data sets using AWS services such as Glue and Lambda functions.
Utilization of AWS SNS to send emails and alerts
Authoring ETL processes using Python and PySpark
ETL process monitoring using CloudWatch events
Connecting with different data sources like S3 and validating data using Athena.
Experience in CI/CD using GitHub Actions
Proficiency in Agile methodology
Extensive working experience with Advanced SQL and a complex understanding of SQL.
Competencies / Experience:
Deep technical skills in AWS Glue (Crawler, Data Catalog)5 years.
Hands-on experience with Python and PySpark3 years.
PL/SQL experience3 years
CloudFormation and Terraform2 years
CI/CD GitHub actions1 year
Experience with BI systems (PowerBI, Tableau)1 year
Good understanding of AWS services like S3, SNS, Secret Manager, Athena, and Lambda2 years
Job Classification
Industry: IT Services & Consulting Functional Area / Department: Engineering - Software & QA Role Category: Software Development Role: Data Engineer Employement Type: Full time