Senior Data Engineer
- Atyeti
- 8 - 13 years
- Pune
- 16 days ago
- Email to a friend
- Report this job
Job Description
Key Responsibilities:
Design, build, and maintain scalable ETL data pipeline using ADLS Gen2, Azure Blob storage, Azure Databricks, Python, Azure SQL Database and Azure Cosmos DB.
Perform deep database performance tuning, query optimization, and indexing strategies across Azure SQL Database and Cosmos DB
Lead database migration initiatives, including strategic migration from Azure SQL Database and CosmosDB to PostgreSQL
Implement efficient data models and manage data storage solutions using ADLS Gen2 and real time data streaming using ADF, Spark and Confluent Kafka.
Develop and maintain data quality checks, monitoring, and alerting systems. Ensure data solutions comply with regulatory and security standards (e.g., PCI DSS)
Optimize batch processing workflows for intra-day and end-of-day banking operations
Required Qualifications:
10 years of hands-on experience in data engineering with strong database expertise.
At least 5 years of experience in implementing operational data store (ODS) for the retail banking space, which captures the multiple cross system transactions, providing high throughput and low latency solution using RDBMS or NoSQL Technologies.
Should have hands-on experience implementing solutions using ADLS Gen2, Azure Blob storage, Azure Databricks, Azure SQL Database and Azure Cosmos DB.
Should have hand on experience in writing queries using the Normalized Data model for the retail banking space. Expert-level skills in database performance tuning, query optimization, and execution plan analysis
Proven experience with large-scale database migrations (Azure SQL Database to PostgreSQL preferred)
Schema Migration Tools like Liquibase - Hands-on experience database migration tools managing version history and applying changes reliably. And managing schema changes with versioning, tracking, and rollback capabilities.
Strong programming skills in Python for data processing and automation
Experience with data modeling, ETL/ELT processes, and data warehouse concepts.
Excellent problem-solving skills with ability to diagnose and resolve complex data pipeline issues Your expertise
Can-do attitude with eagerness to tackle challenging problems
Strong communication skills for technical discussions and solution proposals.
Collaborate with cross-functional teams to resolve complex data-related performance issues and ability to mentor junior engineers.
Azure platform engineering using PowerShell, ARM Templates, Cloud Automation Frameworks.
Familiarity with Azure DevOps, GitLab, and metadata management tools (Data Governance, MDM, Lineage).
Bachelor or master degree or equivalent in Computer Science major. Position emphasize the need for strong communication, problem-solving skills, and the ability to work in a dynamic, fast-paced environment building mission-critical banking infrastructure.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: AtyetiLocation(s): Pune
Keyskills: Azure Data Bricks Python SQL Apache Spark
Similar positions
Atyeti
In the dynamic landscape of IT contingent workforce and training solutions in India, Diensten Tech Limited stands out as a leading and organized player, providing unparalleled services to esteemed clients.\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nWith a robust business foundation, the company has garnered a...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in