Gcp + Big Query + Python / Pyspark + Etl Professional
- Virtusa
- 7 - 12 years
- Hyderabad
- 5 months ago
- Email to a friend
- Report this job
Job Description
Job description
Design and develop robust ETL pipelines using Python, PySpark, and GCP services.Build and optimize data models and queries in BigQuery for analytics and reporting.Ingest, transform, and load structured and semi-structured data from various sources.Collaborate with data analysts, scientists, and business teams to understand data requirements.Ensure data quality, integrity, and security across cloud-based data platforms.Monitor and troubleshoot data workflows and performance issues.Automate data validation and transformation processes using scripting and orchestration tools.Required Skills & Qualifications:Hands-on experience with Google Cloud Platform (GCP), especially BigQuery.Strong programming skills in Python and/or PySpark.Experience in designing and implementing ETL workflows and data pipelines.Proficiency in SQL and data modeling for analytics.Familiarity with GCP services such as Cloud Storage, Dataflow, Pub/Sub, and Composer.Understanding of data governance, security, and compliance in cloud environments.Experience with version control (Git) and agile development practices. Qualification Design and develop robust ETL pipelines using Python, PySpark, and GCP services.Build and optimize data models and queries in BigQuery for analytics and reporting.Ingest, transform, and load structured and semi-structured data from various sources.Collaborate with data analysts, scientists, and business teams to understand data requirements.Ensure data quality, integrity, and security across cloud-based data platforms.Monitor and troubleshoot data workflows and performance issues.Automate data validation and transformation processes using scripting and orchestration tools.Required Skills & Qualifications:Hands-on experience with Google Cloud Platform (GCP), especially BigQuery.Strong programming skills in Python and/or PySpark.Experience in designing and implementing ETL workflows and data pipelines.Proficiency in SQL and data modeling for analytics.Familiarity with GCP services such as Cloud Storage, Dataflow, Pub/Sub, and Composer.Understanding of data governance, security, and compliance in cloud environments.Experience with version control (Git) and agile development practices.
Job Classification
Industry: BankingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data Platform EngineerEmployement Type: Full timeContact Details:
Company: VirtusaLocation(s): Hyderabad
Keyskills: python pyspark sql data modeling bigquery data validation version control microsoft azure cloud platform job description microservices spring boot etl pipelines git java gcp spark cloud storage data governance agile etl aws process transformation
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in