Data Engineer - Azure - CodeCraft Technologies @ Codecraft Technologies

Experience: 4-7 Years

Location: Bangalore

No. of Positions: 01

Role Description:

As a Data Engineer Azure, you will be responsible for end-to-end design and operation of enterprise-scale data pipelines and infrastructure using Azure Databricks and related services. You will ensure data availability, reliability, and quality across HR, Procurement, and Finance functions while applying software engineering best practices such as modular design, TDD, and CI/CD automation.

You will collaborate with business stakeholders, architects, ML/AI engineers, and Power BI developers to align delivery with business outcomes, operating within an agile team structure.

Key Responsibilities:

Data Pipeline Engineering: Design and operate scalable, high-performance data pipelines using Azure Databricks and Azure ecosystem services.

Code as Assets: Develop all artifacts as version-controlled code (Git) with CI/CD automation.

Data Reliability & Quality: Ensure availability, integrity, and quality of data supporting analytics in HR, Procurement, and Finance.

Analytical Assets: Build data products and assets leveraging software engineering best practices.

Engineering Practices: Apply modular design, TDD, reusable code patterns, and performance optimization.

Collaboration: Work closely with architects, ML/AI engineers, functional IT, Power BI developers, and business stakeholders to align solutions with requirements.

Agile Contribution: Actively participate in sprint planning, retrospectives, and demos within an enterprise agile product team.

Monitoring & Optimization: Monitor and improve data infrastructure performance, proactively identifying bottlenecks and optimization opportunities.

Required Skills & Qualifications

Strong experience with Azure Databricks and the broader Azure Data ecosystem (ADF, Synapse, Data Lake, Event Hub).

Proficiency in Python/Scala/SQL for data engineering.

Hands-on experience in CI/CD automation, Git version control, and test-driven development (TDD).

Solid understanding of data modeling, ETL/ELT pipelines, and modular design patterns.

Ability to build analytical data products using engineering-first principles.

Experience working in agile, enterprise-scale environments.

Strong problem-solving and debugging skills.

Preferred Qualification

Exposure to AI/ML pipelines integration in data workflows.

Experience with Power BI or equivalent BI platforms.

Knowledge of DevOps practices in a cloud-first environment.

Microsoft Azure Data Engineer Associate certification (or equivalent).

Desired Profile

A pragmatic engineer passionate about building reliable, reusable, and scalable data solutions.

Collaborative and able to work seamlessly with business, IT, and analytics stakeholders.

Self-driven with strong ownership, accountability, and problem-solving orientation.

Excited to work in a fast-paced, agile product team structure.