Azure Data Engineer | Python, Pyspark, Databricks,cloud Data Pipelines
- Synechron
- 10 - 15 years
- Pune
- 3 days ago
- Email to a friend
- Report this job
Job Description
Azure Data Engineer | Python, PySpark, Databricks, Cloud Data Pipelines, Banking Domain
Job Summary
Synechron is seeking a highly skilled Associate Specialist - Python Data Engineer to join our data engineering team. In this role, you will design, develop, and optimize scalable data pipelines and data solutions using Python, PySpark, and Azure Databricks technologies. Your efforts will enable enterprise analytics, support data-driven decision-making, and contribute to the organizations digital transformation objectives. You will collaborate with cross-functional teams to ensure the delivery of secure, high-performance, and compliant data architectures aligned with business goals.
Software Requirements
- Required: Python (version 3.7+), PySpark, Azure Databricks, SQL, Azure Data Factory, ADLS Gen2, Azure Synapse, Azure Key Vault
- Preferred: Databricks Workflows, REST APIs, Unity Catalog, Docker, Git, Power BI, Tableau
- Experience level: Proven hands-on experience with enterprise-grade cloud data platforms, performance tuning, and automation tools
Overall Responsibilities
- Responsible for designing and building scalable ETL/ELT pipelines and data marts within the Azure cloud environment
- Optimize data workflows for maximum performance, cost-effectiveness, and security using best practices in Delta Lake and distributed computing
- Collaborate with data architects, subject matter experts, and business teams to translate business requirements into technical data solutions
- Ensure data quality, lineage, security, and compliance through the implementation of validation and governance frameworks
- Contribute to team best practices, perform code reviews, and mentor junior engineers
- Engage in Agile methodologies, including sprint planning, backlog grooming, and continuous delivery
- Monitor systems, troubleshoot issues, and propose improvements to enhance data platform efficiency
Technical Skills (By Category)
- Programming Languages:
- Required: Python, PySpark, Advanced SQL
- Preferred: REST API scripting, automation via Python
- Databases/Data Management:
- Required: Delta Lake, relational and dimensional modeling, star schema design, SCDs
- Preferred: Unity Catalog, metadata management tools
- Cloud Technologies:
- Required: Azure Databricks, Azure Data Factory, ADLS Gen2, Azure Synapse, Key Vault
- Preferred: Azure DevOps, containerization (Docker)
- Frameworks and Libraries:
- Required: PySpark, Delta Lake, CI/CD pipeline integration
- Preferred: APIs, Unity Catalog, custom Python libraries
- Development Tools Methodologies:
- Required: Agile/Scrum, Git version control, continuous integration/delivery workflows
- Security Protocols:
- Knowledge of data security best practices, encryption, role-based access, and audit logging
Experience Requirements
- Minimum of 10 years of experience in Data Engineering or equivalent roles with a focus on cloud data platforms
- Strong background in designing and deploying large-scale data pipelines, data lakes, and data marts in cloud environments
- Experience working within banking, financial services, or regulated industries is highly preferred
- Alternative experience: Extensive enterprise data architecture expertise with a focus on compliance and security
Day-to-Day Activities
- Develop, test, and deploy scalable data pipelines and analytics data sets
- Monitor pipeline performance, troubleshoot issues, and optimize resource utilization
- Collaborate frequently with technical and business stakeholders to refine data requirements
- Conduct data validation, establish governance procedures, and ensure regulatory compliance
- Participate in team meetings, project planning, and code review sessions
- Document technical designs, processes, and best practices for knowledge sharing
Qualifications
- Bachelors degree in Computer Science, Information Technology, Data Science, or a related field; advanced degrees are a plus
- Relevant certifications such as Azure Data Engineer, Databricks Certification, or equivalent are preferred
- Commitment to ongoing learning and staying current with emerging data engineering technologies and best practices
- Experience with financial industry data standards and regulatory requirements is advantageous
Professional Competencies
- Strong analytical and problem-solving skills with attention to detail
- Ability to collaborate across teams and mentor junior staff
- Clear communication skills for technical and non-technical stakeholders
- Adaptability to evolving technologies and project requirements
- Focused on delivering sustainable, scalable, and high-quality data solutions
- Effective time management and ability to prioritize tasks to meet deadlines
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: SynechronLocation(s): Pune
Keyskills: data engineer Pyspark Azure Data Pipelines Compliance Architecture Power Bi Banking Tableau Devops Sql Data Science Data Quality Git Docker Cloud Agile Learning And Development Scrum Databricks Sprint Etl Advanced Sql Python
Similar positions
Synechron
Headquartered in New York and with 18 offices around the world, Synechron is helping global financial services and insurance companies embrace the most cutting-edge innovations to evolve their businesses. Synechron uniquely delivers these firms an end-to-end Digital, Consulting and Technology capabi...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in