Google Data Engineer
- Tata Consultancy
- 7 - 10 years
- Kolkata
- 5 days ago
- Email to a friend
- Report this job
Job Description
Google Cloud Platform (GCP) Data Engineer to design, develop, and manage scalable and efficient data pipelines, integrations, and analytics solutions. The ideal candidate will leverage GCP's suite of tools to optimize data processes and enable business intelligence.
Key Responsibilities:
Data Pipeline Development:
- Design and implement ETL/ELT workflows using GCP services such as Dataflow, BigQuery, and Cloud Composer (Apache Airflow).
- Develop real-time and batch data processing pipelines.
Data Integration and Management:
- Integrate data from various structured and unstructured sources using APIs, Pub/Sub, and Cloud Storage.
- Perform data modeling and database design for optimal performance.
Data Analytics and Optimization:
- Utilize BigQuery for building and optimizing data warehouses.
- Design solutions to improve data quality, reliability, and efficiency.
Cloud Infrastructure:
- Set up and manage GCP environments including IAM, Cloud Storage, and networking.
- Optimize compute resources using services like Cloud Run, Kubernetes, and DataProc.
Collaboration and Reporting:
- Work with cross-functional teams to understand data requirements.
- Provide dashboards and insights using BI tools like Looker or third-party platforms.
Documentation and Best Practices:
- Maintain technical documentation for data workflows.
- Ensure compliance with security and privacy standards.
Required Skills:
Technical Proficiency in GCP Services:
- BigQuery, Dataflow, Pub/Sub, Cloud Storage, DataProc, Cloud Composer, and AI/ML tools.
Programming and Querying:
- Proficient in SQL, Python, and Java/Scala (for Dataflow or Spark).
- Experience with BigQuery SQL optimizations.
Data Processing and Analytics:
- Strong experience in ETL/ELT workflows.
- Knowledge of real-time streaming technologies and tools.
Cloud Infrastructure:
- Experience in setting up GCP environments, networking, IAM policies, and CI/CD pipelines.
Tools and Frameworks:
- Hands-on experience with Airflow, Terraform, and CI/CD tools (e.g., Jenkins, GitHub Actions).
- Familiarity with containerization using Docker and Kubernetes.
Data Management:
- Expertise in data warehousing, data modeling, and database performance tuning.
Soft Skills:
- Strong problem-solving and analytical skills.
- Ability to communicate complex ideas to technical and non-technical stakeholders.
- Team collaboration and adaptability.
Preferred Qualifications:
- GCP Professional Data Engineer certification.
- Experience with AI/ML tools in GCP (e.g., AutoML, Vertex AI).
- Familiarity with data governance and security standards (e.g., GDPR, HIPAA).
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Software Development - OtherEmployement Type: Full timeContact Details:
Company: Tata ConsultancyLocation(s): Kolkata
Keyskills: Data Pipeline Data Analytics Data Integration
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in