Azure Data Engineer @ Infogain

Home > Software Development

Azure Data Engineer

Infogain
5 - 8 years
Noida, Gurugram
2 months ago
Email to a friend
Report this job

Job Description

Analyze existing Hadoop, Pig, and Spark scripts from Dataproc and refactor them into Databricks-native PySpark.
Implement data ingestion and transformation pipelines using Delta Lake best practices.
Apply conversion rules and templates for automated code migration and testing.
Conduct data validation between legacy and migrated environments (schema, count, and data-level checks).
Collaborate on developing AI-driven tools for code conversion, dependency extraction, and error remediation.
Ensure best practices for code versioning, error handling, and performance optimization.
Participate in UAT, troubleshooting, and post-migration validation activities.

Technical Skills

Core: Python, PySpark, SQL
Databricks: Delta Lake, Unity Catalog, Databricks Workflows, MLflow (basic understanding)
GCP: Dataproc, BigQuery, GCS, Composer/Airflow, Cloud Functions
Data Engineering: Hadoop, Hive, Pig, Spark SQL
Automation: Experience with migration utilities or AI-assisted code transformation tools
CI/CD: Git, Jenkins, Terraform (preferred)
Validation: Data comparison utilities (Delta-to-Delta, DataFrame diffing, schema validation)

Preferred Experience

5-8 years in data engineering or big data application development.
Hands-on experience migrating Spark or Hadoop workloads to Databricks.
Familiarity with Delta architecture , data quality frameworks , and GCP cloud integration .
Exposure to GenAI-based tools for automation or code refactoring is a plus.

EXPERIENCE

6-8 Years

Job Classification

Industry: IT Services & Consulting
Functional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Data Engineer
Employement Type: Full time

Contact Details:

Company: Infogain
Location(s): Noida, Gurugram

+ View Contact

Login

Candidates can login here to view contacts and apply.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach Resume Max 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Candidates are expected to provide most recent and accurate profile information, inappropriate content is strictly prohibited!

Keyskills: hive continuous integration python scala pyspark microsoft azure apache pig code versioning tools machine learning data engineering application development sql dataproc gen git automation spark gcp jenkins terraform hadoop bigquery sqoop big data

Fraud Alert to job seekers!

₹ Not Disclosed

Job application

We will notify the employer with your details. You can also attach a resume or a cover letter.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach ResumeMax 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Similar positions

Hcl Is Hiring Java Bigdata Developer@bangalore/hyderabad

HCLTech

9 - 14 years

Hyderabad

21 hours ago

₹ Not Disclosed

Custom Software Engineer

Accenture HR Aditi

2 - 5 years

Coimbatore

21 hours ago

₹ Not Disclosed

Gen AI Engineer

Cognizant

9 - 12 years

Chennai

23 hours ago

₹ Not Disclosed

Custom Software Engineer

Accenture

3 - 8 years

Pune

1 day ago

₹ Not Disclosed

Infogain

A global digital engineering company delivering technology solutions that accelerate business outcomes. It specializes in cloud, data, AI, and experience-led transformation for enterprises across industries.

Azure Data Engineer @ Infogain

Home > Software Development