Data Engineering Lead - Python & AWS @ Centric Consulting

Home > Software Development

Data Engineering Lead - Python & AWS

Centric Consulting
5 - 10 years
Kolkata
10 days ago
Email to a friend
Report this job

Job Description

We are looking for a Data Engineering Lead with deep expertise in AWS, Python , and modern data engineering practices to architect and deliver scalable, secure, and efficient data solutions. In this role, you ll lead the design and development of ETL pipelines, implement robust data warehousing solutions, and support machine learning workflows across various business domains.

Your strong understanding of ETL and Data Warehousing concepts , including Slowly Changing Dimensions (SCD) and Medallion Architecture , along with proficiency in SQL and AWS-native tools, will be key to success.

Key Responsibilities

Data Pipeline & Architecture

Design, develop, and maintain scalable ETL pipelines leveraging AWS services such as Glue , S3 , Lambda , and Athena .
Hands on Python and Pyspark coding experience.
Implement Medallion Architecture (bronze, silver, gold layers) to organize and optimize data for both operational and analytical use cases.
Apply best practices in data modeling and data warehousing , including handling SCD Type 1 and Type 2 transformations.

Data Management & Quality

Ensure data accuracy, completeness, and reliability across batch and near-real-time workflows.
Create robust error handling, logging, and monitoring processes for pipeline operations.
Write and optimize complex SQL queries for data transformations, validations, and reporting use cases.

Collaboration & Delivery

Partner with data engineers , analysts , and product teams to enable data access and support analytical models and dashboards.
Participate in code reviews , contribute to best practices , and ensure high engineering standards.
Troubleshoot and resolve data pipeline and infrastructure issues in production environments.

Must-Have Qualifications

5+ years of hands-on experience in data engineering , particularly with Python , PySpark , and SQL .
Proven experience building ETL pipelines using AWS services (Glue, Lambda, S3, Athena, Step Functions).
Strong knowledge of Data Warehousing concepts , SCD Types (Type 1, Type 2) , and Medallion Architecture .
Solid understanding of data modeling , partitioning , and performance optimization in big data environments.
Excellent communication skills with the ability to explain technical topics to non-technical stakeholders.

Nice-to-Have Skills

Experience with DBT for data transformation and testing.
Hands-on exposure to Databricks and the Lakehouse architecture .
Familiarity with CI/CD for data pipelines and infrastructure-as-code using tools like Terraform or CloudFormation.

Job Classification

Industry: Recruitment / Staffing
Functional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Data Engineer
Employement Type: Full time

Contact Details:

Company: Centric Consulting
Location(s): Kolkata

+ View Contact

Login

Candidates can login here to view contacts and apply.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

500 characters remaining

Attach Resume Max 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Candidates are expected to provide most recent and accurate profile information, inappropriate content is strictly prohibited!

Keyskills: Data management Architecture Data modeling Coding Data warehousing AWS Monitoring Engineering Lead SQL Python

Fraud Alert to job seekers!

₹ Not Disclosed

Job application

We will notify the employer with your details. You can also attach a resume or a cover letter.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach ResumeMax 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Similar positions

Lead Developer

Cognizant

5 - 8 years

Noida, Gurugram

14 hours ago

₹ Not Disclosed

Software Engineer III - UI

JPMorgan Chase Bank

0 - 5 years

Bengaluru

15 hours ago

₹ Not Disclosed

M&C Delivery professional - B

Capgemini

4 - 7 years

Kolkata

16 hours ago

₹ Not Disclosed

Sr. Business Analyst - For Bangalore

Tech Mahindra

8 - 13 years

Bengaluru

17 hours ago

₹ Not Disclosed

Centric Consulting

Centric Consulting is an international management consulting firm with unmatched in-house expertise in business transformation, hybrid workplace strategy, technology implementation and adoption. The firm has established a reputation for combining the benefits of experience, flexibility and cost effi...

Data Engineering Lead - Python & AWS @ Centric Consulting

Home > Software Development