Technical Lead - Azure Databricks @ CitiusTech

As a Technical Lead Azure Databricks , you will be part of an agile team to build healthcare applicationsand implement new features while adhering to the best coding developmentstandards.

Your key responsibilitiesinclude:

• Design and develop ETL/ELT solutions on Azure Databricks,Delta Lake, Spark, and PySpark
• Develop, implement, and deploy large scale data pipelinesempowering machine learning algorithms, insights generation, businessintelligence dashboards, reporting and new data products
• Partner with Client Technology to create and maintain thetechnical architecture of the Enterprise Delta Lake to consolidate data frommany systems into a single source for machine learning and reporting analytics
• Design, build, optimize, and manage modern large scale datapipelines (ETL/ELT processing) to support data integration for analytics,machine learning features, and predictive modeling
• Consume data from a variety of sources (Live streaming fromKafka, Azure Event Hub, RDBMS, APIs, FTPs and other Cloud storage) and formats(Excel, CSV, XML, JSON, Parquet, Unstructured)
• Write advanced/complex SQL with performance tuning andoptimization
• Identify ways to improve data reliability, data integrity, andsystem efficiency and quality

Education

• Engineering Degree BE / BTech / MTech / BCA / MCA / BSc (IT/CS)
• Technical certifications inmultiple technologies are desirable

Required Skills

• Should have minimum 2 yearsof working experience in Databricks
• Strong knowledge of dataarchitecture principles such as Data Lake, Databricks Delta Lake, DataWarehousing, and more
• Must have minimum 2 years of workingexperience in Apache Spark (PySpark/Spark SQL)
• Should have minimum 2 yearsof working experience in Python
• Must have minimum 3 years of workingexperience in SQL along with designing complex data schemas and queryperformance optimization
• Strong understanding of processinglive data streaming from Azure Event Hub
• Must have working experience inat least one of the Cloud platforms, namely, Azure, AWS, or GCP
• Strong understanding of Terraform,Kafka
• Should have working experiencein CI/CD technology
• Must have working experience inGit