Sr. Data Engineer
- Photon
- 8 - 9 years
- Kolkata
- 11 months ago
- Email to a friend
- Report this job
Job Description
- Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
- Develop data pipelines to extract and transform data in near real-time using cloud-native technologies.
- Implement data validation and quality checks to ensure accuracy and consistency.
- Monitor system performance, troubleshoot issues, and implement optimizations to enhance reliability and efficiency .
- Collaborate with business users, analysts, and other stakeholders to understand data requirements and deliver tailored solutions.
- Document technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system documentation.
- Provide technical guidance and support to team members and stakeholders as needed.
- 8+ years of work experience.
- Proficiency in writing complex SQL queries on MPP systems (Snowflake/Redshift).
- Experience in Databricks and Delta tables.
- Data engineering experience with Spark/Scala/Python.
- Experience in Microsoft Azure stack (Azure Storage Accounts, Data Factory, and Databricks).
- Experience in Azure DevOps and CI/CD pipelines.
- Working knowledge of Python.
- Comfortable participating in 2-week sprint development cycles.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Machine LearningRole: Data EngineerEmployement Type: Full timeContact Details:
Company: PhotonLocation(s): Kolkata+ View Contact
Keyskills: SQL queries Data validation spark Delta devops SCALA microsoft azure Troubleshooting Monitoring Python
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in