Data - Senior Engineer
- Iris Software
- 5 - 7 years
- Noida, Gurugram
- 3 days ago
- Email to a friend
- Report this job
Job Description
Job Duties Responsibilities
- Design, develop, and optimize complex data pipelines and transformation processes using Snowflake, dbt, and AWS services.
- Databricks is Mandatory
- Implement and manage data integration workflows using Fivetran to ensure timely and accurate data ingestion from various sources.
- Develop and maintain scalable data models and schemas in Snowflake, ensuring they meet performance and business requirements.
- Monitor and fine-tune the performance of data pipelines, queries, and data models to ensure optimal efficiency and cost-effectiveness.
- Utilize Snowflakes features, such as Time Travel, Zero-Copy Cloning, and Data Sharing, to enhance data management and performance.
- Leverage AWS services, such as AWS Lambda, S3, and Glue, to build and manage serverless data processing workflows and data storage solutions.
- Implement data security measures and ensure compliance with data privacy regulations and organizational policies.
- Troubleshoot and resolve complex data issues, including data sync errors, performance bottlenecks, and integration challenges.
- Provide support for data-related incidents and ensure effective resolution of production issues.
- Collaborate with data analysts, and other stakeholders to understand data needs and deliver effective solutions.
- Document data processes, models, and workflows, ensuring clear communication and knowledge sharing across teams.
- Independently assess situations, apply sound judgment and discretion, and make decisions on matters of significant impact without direct supervision
Basic Qualifications
- Bachelors Degree in Computer Science, Data Science, Information Technology, Engineering, or a related field. Snowflake certification a plus.
- 5-7 years of experience in data engineering, with a strong focus on data transformation, integration, and cloud-based data solutions
- Experience in writing complex SQL queries
- 5 years of experience on Snowflake platform.
- Experience in data modeling and writing data transformation with dbt platform, Tableau data prep, etc.
- Experience in building connector with Fivetran platform.
- Experience in writing test automation scripts.
- Experience using/understanding Snowflake Warehouses, reader accounts, SSO Setup, and data masking policies.
- Implemented Change Data Capture (CDC) and Change Data Tracking mechanisms in data pipelines.
- Experience designing and optimizing data ingestion pipelines to handle large-volume datasets.
- Experience writing complex data transformation logic using Advanced SQL query skills such as complex joins, sub-queries, grouping, aggregation, and stored procedures.
- Experience with programming languages, especially Python and Java.
- Designed and/or managed data warehousing concepts, methodologies, and best practices.
- Excellent oral and written communication skills including the ability to speak in front of large groups.
Preferred Qualifications
- Data Architecture Data modeling experience in Clinical trials domain / Life Sciences.
- Knowledge of data management tools and process; Data governance tools is a plus.
- Working experience with version control platforms, e.g. Github, and agile methodologies and supporting tools JIRA.
- Ability to handle stress and interact with others in a professional manner.
- Comfort working in a geographically distributed team-based environment.
Mandatory Competencies
Data Science and Machine Learning - Data Science and Machine Learning - Databricks
Cloud - Cloud - Snowflake
Cloud - AWS - AWS S3, S3 glacier, AWS EBS
Cloud - AWS - Tensorflow on AWS, AWS Glue, AWS EMR, Amazon Data Pipeline, AWS Redshift
Cloud - AWS - AWS Lambda,AWS EventBridge, AWS Fargate
Database - Oracle - PL/SQL Packages
Tech - Agile Methodology
DevOps/Configuration Mgmt - DevOps/Configuration Mgmt - GitLab,Github, Bitbucket
Development Tools and Management - Development Tools and Management - JIRA
Data Science and Machine Learning - Data Science and Machine Learning - Tableau
Database - PostgreSQL - PostgreSQL
Database - Sql Server - SQL Packages
Data Science and Machine Learning - Data Science and Machine Learning - Python
Programming Language - Java - Core Java (java 8+)
Beh - Communication and collaboration
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Business Intelligence & AnalyticsRole: Business AnalystEmployement Type: Full timeContact Details:
Company: Iris SoftwareLocation(s): Noida, Gurugram+ View Contact
Keyskills: compliance data capture data warehousing bitbucket sql tensorflow lambda java postgresql data science redshift devops architecture jira s3 snowflake python github cdc databricks machine learning sql server gitlab agile aws
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in