Design, develop, test, deploy and maintain large-scale data pipelines using Data Bricks on Microsoft Azure platform.
Collaborate with cross-functional teams to gather requirements and design solutions for complex data processing needs.
Develop scalable and efficient ETL processes to extract insights from various sources of structured and unstructured data.
Troubleshoot issues related to data quality, performance, security, and compliance in the developed solutions.
Job Requirements :
10-15 years of experience in Data Engineering with expertise in Scala programming language.
Strong understanding of Big Data technologies such as Hadoop ecosystem (HDFS, MapReduce) and Spark (Scala).
Experience working with cloud-based platforms like Microsoft Azure including storage services (Blobs), compute services (Virtual Machines), database services (SQL Database).
Proficiency in developing ETL processes using Data Bricks framework.
Job Classification
Industry: InternetFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full time