Associate Data Architect I- Azure/snowflake & Aws Migration
- UST
- 9 - 12 years
- Thiruvananthapuram
- 4 months ago
- Email to a friend
- Report this job
Job Description
Tech Lead - Azure/Snowflake & AWS Migration
Key Responsibilities
- Design and develop scalable data pipelines using Snowflake as the primary data platform, integrating with tools like Azure Data Factory, Synapse Analytics, and AWS services.
- Build robust, efficient SQL and Python-based data transformations for cleansing, enrichment, and integration of large-scale datasets.
- Lead migration initiatives from AWS-based data platforms to a Snowflake-centered architecture, including:
o Rebuilding AWS Glue pipelines in Azure Data Factory or using Snowflake-native ELT approaches.
o Migrating EMR Spark jobs to Snowflake SQL or Python-based pipelines.
o Migrating Redshift workloads to Snowflake with schema conversion and performance optimization.
o Transitioning S3-based data lakes (Hudi, Hive) to Snowflake external tables via ADLS Gen2 or Azure Blob Storage.
o Redirecting Kinesis/MSK streaming data to Azure Event Hubs, followed by ingestion into Snowflake using Streams & Tasks or Snowpipe.
- Support database migrations from AWS RDS (Aurora PostgreSQL, MySQL, Oracle) to Snowflake, focusing on schema translation, compatibility handling, and data movement at scale.
- Design modern Snowflake lakehouse-style architectures that incorporate raw, staging, and curated zones, with support for time travel, cloning, zero-copy restore, and data sharing.
- Integrate Azure Functions or Logic Apps with Snowflake for orchestration and trigger-based automation.
- Implement security best practices, including Azure Key Vault integration and Snowflake role-based access control, data masking, and network policies.
- Optimize Snowflake performance and costs using clustering, multi-cluster warehouses, materialized views, and result caching.
- Support CI/CD processes for Snowflake pipelines using Git, Azure DevOps or GitHub Actions, and SQL code versioning.
- Maintain well-documented data engineering workflows, architecture diagrams, and technical documentation to support collaboration and long-term platform maintainability.
Required Qualifications
- 9+ years of data engineering experience, with 3+ years on Microsoft Azure stack and hands-on Snowflake expertise.
- Proficiency in:
o Python for scripting and ETL orchestration
o SQL for complex data transformation and performance tuning in Snowflake
o Azure Data Factory and Synapse Analytics (SQL Pools)
- Experience in migrating workloads from AWS to Azure/Snowflake, including services such as Glue, EMR, Redshift, Lambda, Kinesis, S3, and MSK.
- Strong understanding of cloud architecture and hybrid data environments across AWS and Azure.
- Hands-on experience with database migration, schema conversion, and tuning in PostgreSQL, MySQL, and Oracle RDS.
- Familiarity with Azure Event Hubs, Logic Apps, and Key Vault.
- Working knowledge of CI/CD, version control (Git), and DevOps principles applied to data engineering workloads.
Preferred Qualifications
- Extensive experience with Snowflake Streams, Tasks, Snowpipe, external tables, and data sharing.
- Exposure to MSK-to-Event Hubs migration and streaming data integration into Snowflake.
- Familiarity with Terraform or ARM templates for Infrastructure-as-Code (IaC) in Azure environments.
- Certification such as SnowPro Core, Azure Data Engineer Associate, or equivalent.
Required Skills
Azure,AWS REDSHIFT,Athena,Azure Data Lake
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: DBA / Data warehousingRole: Data warehouse Architect / ConsultantEmployement Type: Full timeContact Details:
Company: USTLocation(s): Thiruvananthapuram
Keyskills: continuous integration orchestration data glue amazon redshift ci/cd azure data factory snowpipe sql analytics scripting tables devops etl snowflake cd python streams microsoft azure data engineering environment cloud architecture microsoft stack athena aws data sharing
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in