Design, develop, test, deploy and maintain large-scale data pipelines using Airflow on Google Cloud Platforms (GCP).
Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
Develop complex SQL queries to extract insights from large datasets stored in PostgreSQL databases.
Troubleshoot issues related to pipeline failures or errors in PySpark jobs.
Job Requirements :
4-9 years of experience in Data Engineering with expertise in GCP technologies such as BigQuery, Pub/Sub, Storage etc.
Strong proficiency in Python programming language with experience working with libraries like NumPy, Pandas etc.
Experience with Airflow workflow management tool for scheduling tasks at scale.
Job Classification
Industry: InternetFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full time