Lead Data Engineer
- Financial Services
- 8 - 12 years
- Hyderabad
- 3 months ago
- Email to a friend
- Report this job
Job Description
Data Engineering Lead
About the Role
We are searching for a strategic, results-driven Data Engineering Lead to join our Data Science software engineering team. The ideal candidate will possess a robust technical background in data engineering, extensive hands-on experience with modern data architectures, and a proven ability to mentor and lead high-performing engineering teams. As a thought leader, you will design and implement scalable data solutions that empower our data scientists and drive impactful insights across the organization.
Key Responsibilities
- Lead, mentor, and grow a team of data engineers, fostering technical excellence and professional development.
- Architect, develop, and maintain advanced ELT data pipelines using state-of-the-art tools and best practices.
- Collaborate closely with data scientists to understand analytical requirements and translate them into scalable, reliable data solutions.
- Oversee the management and optimization of distributed, column-oriented databases such as Vertica and Snowflake to ensure high performance and data integrity.
- Drive the implementation and continuous improvement of data workflow orchestration with tools like Apache Airflow.
- Conduct root cause analysis and lead troubleshooting efforts to resolve complex data issues, ensuring availability and quality of critical datasets.
- Champion best practices for data governance, security, and compliance throughout all engineering processes.
- Document data processes, workflows, and architectural decisions for transparency and knowledge sharing.
- Promote innovation by evaluating emerging technologies and integrating them into existing data platforms as appropriate.
Required Qualifications
- Bachelors degree in computer science, Engineering, or a related field.
- 8+ years of progressive experience in data engineering, with at least 2 years in a technical leadership role.
- Expertise in Python and SQL for large-scale data manipulation and transformation.
- Extensive experience with cloud-based data platforms and services, particularly AWS.
- Deep understanding of data warehousing, dimensional modelling, and analytics frameworks.
- Hands-on experience with orchestration and workflow automation tools such as Apache Airflow and Jenkins.
- Proficiency in distributed, column-oriented databases (e.g., Vertica, Snowflake) and familiarity with CI/CD practices using Git or similar tools.
- Demonstrated ability to solve complex analytical problems and deliver high-quality, reliable data solutions at scale.
- Outstanding communication, collaboration, and organizational skills, with a track record of cross-functional partnership.
Preferred Qualifications
- Masters degree in computer science, Engineering, or a related discipline.
- Experience implementing data governance, security, and compliance at scale.
- Knowledge of Apache Spark for distributed data processing and Apache Kafka for real-time data streaming.
- Background in supporting data science workflows and familiarity with machine learning data needs.
Join us to lead our data engineering efforts and enable the next generation of data science innovation.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: OtherRole Category: OtherRole: OtherEmployement Type: Full timeContact Details:
Company: Financial ServicesLocation(s): Hyderabad
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in