Opening For Sr Data Engineer - Azure Databricks Aws
- EXL
- 5 - 10 years
- Bengaluru
- 3 months ago
- Email to a friend
- Report this job
Job Description
Key Responsibilities:
- Lead the design, development, and deployment of advanced data pipelines and analytical workflows leveraging Azure Databricks, PySpark, and other big data technologies.
- Implement and optimize ETL/ELT processes for data ingestion, transformation, and loading from diverse sources into data lakes and data warehouses on Azure and AWS.
- Develop and maintain data models, ensuring data quality, integrity, and governance.
- Utilize Python and SQL extensively for data manipulation, scripting, automation, and complex data analysis.
- Work with AWS services (e.g., S3, Glue, Redshift, Lambda) and Azure services (e.g., Data Lake, Data Factory, Synapse Analytics) to build and manage data infrastructure.
- Monitor, troubleshoot, and optimize data pipeline performance and reliability, ensuring data availability and accuracy.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
- Implement robust data validation and testing frameworks to ensure data quality and compliance.
- Mentor junior data engineers and provide technical guidance on best practices in data engineering and cloud technologies.
- Contribute to the implementation of CI/CD pipelines for data solutions and infrastructure as code.
Required Skills & Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Data Engineer, with a significant portion in a senior or lead role.
- Expertise in Azure Databricks, including Delta Lake and Apache Spark.
- Strong proficiency in Python and SQL for data engineering tasks.
- Hands-on experience with AWS cloud services for data storage, processing, and analytics.
- Solid understanding of data warehousing concepts, dimensional modeling, and ETL/ELT principles.
- Experience with data governance, data quality, and security best practices.
- Familiarity with CI/CD pipelines and DevOps practices for data solutions.
- Excellent problem-solving, analytical, and communication skills.
- Ability to work independently and as part of a collaborative team
Job Classification
Industry: Analytics / KPO / ResearchFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full timeContact Details:
Company: EXLLocation(s): Bengaluru+ View Contact
Keyskills: Microsoft Azure AWS Data Bricks SQL Python Pyspark
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in