Lead Azure Big Data Engineer
- in3corp
- 3 to 7 Yrs
- Hyderabad
- 12 days ago
- Email to a friend
- Report this job
Job Description
- Skills: Azure Databricks, Data Factory, Python, SQL, PySpark, Big Data, ETL, Architect designing, Senior Azure Data Engineer Location: Hyderabad, Bangalore and Pune Experience: 3-7 Years Work Mode: Hybrid Mandatory Skills: Python, Spark, SQL, Hadoop, Hive, Spark, Azure Databricks, Azure DataFactory, Azure Synapse, DLTs, Unity Catalog, Workflows, Infrastructure as Code (IaC), Orchestration-Airflow, ETL & Data Pipelines. Role Overview We are looking for a Senior Data Engineer with a strong background in Python, Spark, Databricks, and Azure to join our growing team. The ideal candidate will have extensive experience in data processing, working with cloud platforms (specifically Azure), and be able to handle complex data engineering tasks. Key Responsibilities Design, build, and optimize scalable data pipelines for large-scale data processing using Python and Apache Spark. Leverage Azure Databricks to perform big data processing, including working with PySpark to process and analyze large datasets. Utilize Azure Data Factory for seamless data integration and orchestration across cloud environments. Implement and maintain solutions using Azure Data Lake Storage for big data analytics and transformation. Write efficient and reusable Python code, utilizing libraries such as Pandas and NumPy, to process and manipulate data. Work with core Azure data services like Azure SQL Database, Azure Synapse Analytics, Azure Cosmos DB, and Azure Blob Storage. Should have worked with popular tools for Infrastructure as Code (IaC) like Terraform or Biceps in conjunction with CI/CD pipelines for the purpose of provisioning new Azure Infra resources. Architectural experience is plus Design and maintain complex SQL queries for managing and analyzing relational databases. Ensure high-quality data management, integrity, and security across all stages of data processing & analysis. Collaborate with cross-functional teams to understand data requirements and build solutions to meet business needs. Provide mentorship and guidance to junior team members. Key Skills & Qualifications Programming: Proficiency in Python, including knowledge of Pythonic principles, data structures, and algorithms. Hands-on experience with libraries such as Pandas and NumPy. Cloud Technologies: In-depth knowledge of Azure, including Azure SQL Database, Azure Synapse Analytics, Azure Cosmos DB, Azure Data Factory, Azure Blob Storage, and Azure Data Lake Storage. Data Processing: Strong experience with Apache Spark using Azure Databricks and PySpark for large-scale data processing. SQL: Strong skills in SQL for querying and managing relational databases. Ability to write complex and optimized SQL queries. Communication: Excellent communication skills, with the ability to collaborate effectively across teams and present technical solutions to non-technical stakeholders. Problem-Solving: Strong analytical and problem-solving skills, with the ability to troubleshoot and resolve complex data challenges Education: B.Tech in any branch or specialization.,
Employement Category:
Employement Type: Full timeIndustry: IT Services & Consulting
Role Category: Not Specified
Functional Area: Not Specified
Role/Responsibilies: Lead Azure Big Data Engineer
Contact Details:
Company: VidPro ConsultancyLocation(s): Hyderabad
+ View Contact
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
in3corp
in3corp is a growing, entrepreneurial, cash-recovery audit firm based in Boulder, Colorado USA with affiliate office in Lucknow, UP is searching for new talent to fill the position of Audit Analyst in our Lucknow office. This is a full-time position. Our firm serves top global corporations, provid...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in