Azure Data Engineering Lead - Noida - NTT Data

Day to Day job Duties:

• Develop and implement solid, scalable, and flexible solutions in Microsoft Azure.
• Able to strategize an idea, concept and present a blueprint on design and use case delivery from POCs to Enterprise-wide engagements.
• Troubleshoot, diagnose, and work within a team to resolve data challenges: performance, quality, production support, etc.
• Conduct analysis of business processes, data, and systems to understand business data needs
• Knowledge of design strategies for developing scalable, resilient, always-on data lake

Basic Qualifications :

• At least 10 years of experience with the Hadoop ecosystem, Big Data and Cloud technologies
• Expert in Big Data Design and Architecture and able to articulate pros and cons of technologies to a mix of Tech and Business audience.
• Knowledge of design strategies for developing a scalable, resilient, always-on data lake
• Hands-on experience with the Hadoop eco-system - HDFS, Hive, Spark
• Experience with Azure ecosystem is a must - Azure Data Lake, Azure Service Bus, HDInsight, Azure Data Factory, Azure Databricks, Azure Functions, etc.
• Experience in implementing Hadoop Data Lakes - Data storage, partitioning, splitting, file types (Parquet, Avro, ORC) for specific use cases, etc.
• Programming - Java, Python
• Experience with Query languages - SQL, Hive, etc.
• Experience with NoSQL databases - Cosmos DB/MongoDB/Cassandra
• Partner with data analysts, product owners, and data scientists, to better understand requirements, solution designs, finding bottlenecks, resolutions, etc.
• Support/Enhance data pipelines and ETL using heterogeneous sources.
• You will transform data using data mapping and data processing capabilities like Spark, Spark SQL, HiveQL, etc.
• Familiarity with Data ingestion frameworks such as Azure Data Factory, Sqoop, etc.
• Strong development/automation skills. Must be very comfortable with reading and writing Python or Java code.
• Expands and grows data platform capabilities to solve new data problems and challenges.
• Ability to dynamically adapt to conventional big-data frameworks and tools with the use-cases required by the project.
• Experience in agile(scrum) development methodology
• Excellent interpersonal and teamwork skills
• Ability to work in a fast-paced environment and manage multiple simultaneous priorities
• Can-do attitude on problem-solving, quality, and ability to execute

Must-Have Primary Skills :

• Worked on data warehouse & end-user reporting projects with diverse internal and external data sources
• Experience with developing or supporting data orchestration and data lake tools: Spark, Azure Databricks, Azure Data Factory, Azure Service Bus, Azure HDInsight
• Programming - Python, Java/Scala

Nice to Have :

• Masters or bachelor s degree in engineering in Computer Science or Information Technology is desired