Skills
Required Skills: Big data, ETL, DHW, Azure (Databricks)
Roles and responsibilities/What Youll Do
Experience in defining solution architecture
To be able to benchmark systems, analyse system bottlenecks and propose solutions to eliminate them
Able to help program and project managers in the design, planning and governance of implementing projects of any kind;
Translate complex functional and technical requirements into detailed design and development
What Were Looking For
Hands on experience in Hadoop, HDFS & HIVE development and implementation
Hands on Experience in Spark Programming
Good knowledge on Kafka
Good knowledge of database structures, theories, principles, and practices.
Hands on experience in HBase/Cassandra
Good to have experience on SCALA, Python
Translate complex functional and technical requirements into detailed design and development
Implementation experience of Big Data stack in Azure
Design and implement Data Lake solutions on MS Azure
Should have used Data Factory, Azure Data Lake Store, Azure Data Lake Analytics, Data Bricks, SQL DW & Power BI
Experience building scalable data pipeline on MS Azure
Experience in Azure DevOps, CI/CD & Continuous Testing
Knowledge on Streamsets is a plus * Knowledge on Metadata management