Azure Data Tech Lead
- Brillio
- 7 - 11 years
- Hyderabad
- 2 months ago
- Email to a friend
- Report this job
Job Description
Roles and Responsibilities
Key Responsibilities:
Data Pipeline Development: Lead the design, development, and deployment of data pipelines using Azure OneLake, Azure Data Factory, and Apache Spark, ensuring efficient, scalable, and secure data movement across systems.
ETL Architecture: Architect and implement ETL (Extract, Transform, Load) workflows, optimizing the process for data ingestion, transformation, and storage in the cloud.
Data Integration: Build and manage data integration solutions that connect multiple data sources (structured and unstructured) into a cohesive data ecosystem. Use SQL, Python, Scala, and R to manipulate and process large datasets.
Azure OneLake Expertise: Leverage Azure OneLake and Azure Synapse Analytics to design and implement scalable data storage and analytics solutions that support big data processing and analysis.
Collaboration with Teams: Work closely with Data Scientists, Data Analysts, and BI Engineers to ensure that the data infrastructure supports analytical needs and is optimized for performance and accuracy.
Performance Optimization: Monitor, troubleshoot, and optimize data pipeline performance to ensure high availability, fast processing, and minimal downtime.
Data Governance & Security: Implement best practices for data governance, data security, and compliance within the Azure ecosystem, ensuring data privacy and protection.
Leadership & Mentorship: Lead and mentor a team of data engineers, promoting a collaborative and high-performance team culture. Oversee code reviews, design decisions, and the implementation of new technologies.
Automation & Monitoring: Automate data engineering workflows, job scheduling, and monitoring to ensure smooth operations. Use tools like Azure DevOps, Airflow, and other relevant platforms for automation and orchestration.
Documentation & Best Practices: Document data pipeline architecture, data models, and ETL processes, and contribute to the establishment of engineering best practices, standards, and guidelines.
Innovation: Stay current with industry trends and emerging technologies in data engineering, cloud computing, and big data analytics, driving innovation within the team.
Feel free to enter the Interview Room at any time before your scheduled time to check your audio and video settings.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full timeContact Details:
Company: BrillioLocation(s): Hyderabad
Keyskills: Azure Data Factory Azure Synapse Azure Data Warehouse Azure purview Azure Fabric adf Azure Databricks Azure Data Lake SQL Azure
Similar positions
Brillio
Powered by SophosLabs a global threat intelligence and data science team Sophoscloud-native and AI-enhanced solutions secure endpoints (laptops, servers and mobile devices) and networks against evolving cybercriminal tactics and techniques, including automated and active-adversary breaches, ransomwa...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in