bigQuery engineer
- Hsbc
- 7 - 10 years
- Hyderabad
- 22 days ago
- Email to a friend
- Report this job
Job Description
Role & responsibilities
Key Responsibilities:
- Design, develop, and maintain scalable and efficient data pipelines and ETL/ELT workflows using GCP services.
- Architect and implement data warehouse solutions using BigQuery, ensuring optimal performance and cost efficiency.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights.
- Optimize and manage large-scale datasets, ensuring data quality, integrity, and security.
- Develop and enforce best practices for data governance, data modeling, and data lifecycle management.
- Implement streaming data solutions using tools like Dataflow, Pub/Sub, and Apache Beam.
- Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
- Lead the migration of on-premise data systems to GCP, ensuring seamless integration and minimal downtime.
- Mentor and guide junior engineers, fostering a culture of continuous learning and innovation.
- Stay updated with the latest advancements in GCP and data engineering technologies, and recommend improvements to existing systems.
Required Skills and Qualifications:
- 10+ years of experience in data engineering, with a focus on cloud-based solutions.
- Expertise in GCP services, including but not limited to:
- BigQuery
- Dataflow
- Pub/Sub
- Cloud Storage
- Cloud Composer (Airflow)
- Cloud Functions
- Strong proficiency in SQL and experience with BigQuery SQL for complex queries and performance optimization.
- Hands-on experience with Python or Java for building data pipelines and automation.
- Deep understanding of data modeling, data warehousing, and schema design.
- Experience with streaming data processing and tools like Apache Beam or Kafka.
- Familiarity with CI/CD pipelines and version control systems like Git.
- Strong knowledge of data security and compliance standards (e.g., GDPR, HIPAA).
- Proven experience in migrating on-premise data systems to GCP.
- Excellent problem-solving skills and the ability to work in a fast-paced, dynamic environment.
- Strong communication and collaboration skills, with the ability to work effectively with both technical and non-technical stakeholders.
Job Classification
Industry: Investment Banking / Venture Capital / Private EquityFunctional Area / Department: BFSI, Investments & TradingRole Category: Investment Banking, Private Equity & VCRole: Investment Banking, Private Equity & VC - OtherEmployement Type: Full timeContact Details:
Company: HsbcLocation(s): Hyderabad+ View Contact
Keyskills: GCP Bigquery Data Visualization SQL Python Vertex Ai
Fraud Alert to job seekers!
₹ Not Disclosed
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in