Lead Software Engineer
- Mastercard
- 5 - 10 years
- Pune
- 6 months ago
- Email to a friend
- Report this job
Job Description
Title and Summary
Lead Software EngineerJob Summary:As a Lead Software Engineer focused on Data Quality, you will lead the design, development, and deployment of scalable data quality frameworks and pipelines. You will work closely with data engineers, analysts, and business stakeholders to build robust solutions that validate, monitor, and improve data quality across large-scale distributed systems.Key Responsibilities:Lead the design and implementation of data quality frameworks and automated validation pipelines using Python, Apache Spark, and Hadoop ecosystem tools.Develop, deploy, and maintain scalable ETL/ELT workflows using Apache Airflow and Apache NiFi to ensure seamless data ingestion, transformation, and quality checks.
Collaborate with cross-functional teams to understand data quality requirements and translate them into technical solutions.
Define and enforce data quality standards, rules, and monitoring processes.
Perform root cause analysis on data quality issues and implement effective fixes and enhancements.
Mentor and guide junior engineers, conducting code reviews and fostering best practices.
Continuously evaluate and integrate new tools and technologies to enhance data quality capabilities.
Ensure high code quality, performance, and reliability in all data processing pipelines.
Create comprehensive documentation and reports on data quality metrics and system architecture.Required Skills & Experience:* Bachelors or Masters degree in Computer Science, Engineering, or a related field with Data Engineering Experience.
* 5+ years of professional experience in software development, with at least 2 years in a lead or senior engineering role.
* Strong proficiency in Python programming and experience building data processing applications.
* Hands-on expertise with Apache Spark and Hadoop for big data processing.
* Solid experience with workflow orchestration tools like Apache Airflow.
* Experience designing and managing data ingestion and integration pipelines with Apache NiFi.
* Understanding on Data Quality automation, CI/CD, Jenkins, Oracle, Power BI, Splunk
* Deep understanding of data quality concepts, data validation techniques, and distributed data systems.
* Strong problem-solving skills and ability to lead technical discussions.
* Experience with cloud platforms (AWS, GCP, or Azure) is a plus.
* Excellent communication and collaboration skills
Job Classification
Industry: BankingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Technical ArchitectEmployement Type: Full timeContact Details:
Company: MastercardLocation(s): Pune
Keyskills: python data processing spark hadoop big data continuous integration software development oracle data validation workflow orchestration airflow ci/cd microsoft azure power bi quality concepts data quality apache nifi gcp jenkins splunk aws etl
Similar positions
Mastercard
Who is Mastercard?Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks,...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in