Lead Data Engineer AWS - Bengaluru/Bangalore
- Nexplace Info
- 8 - 12 years
- Bengaluru
- 9 months ago
- Email to a friend
- Report this job
Job Description
Lead Data Engineer with AWS
Atleast 2 years of Lead experience is required to apply for this role.
Your JD
- Design and build reusable components, frameworks and libraries at scale to support analytics products.
- Design and implement product features in collaboration with business and Technology stakeholders.
- Anticipate, identify and solve issues concerning data management to improve data quality.
- Clean, prepare and optimize data at scale for ingestion and consumption.
- Drive the implementation of new data management projects and re-structure of the current data architecture.
- Implement complex automated workflows and routines using workflow scheduling tools.
- Build continuous integration, test-driven development and production deployment frameworks.
- Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards.
- Analyze and profile data for the purpose of designing scalable solutions.
- Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues.
- Mentor and develop other data engineers in adopting best practices.
Qualifications:Primary skillset:
- Experience working with distributed technology tools for developing Batch and Streaming pipelines using.
- SQL, Spark, Python [6+ years],
- Airflow [4+ years],
- Scala [4+ years].
- Experience in Cloud Computing, e.g., AWS, GCP, Azure, etc.
- Able to quickly pick up new programming languages, technologies, and frameworks.
- Strong skills building positive relationships across Product and Engineering.
- Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
- Experience with creating/ configuring Jenkins pipeline for smooth CI/CD process for Managed Spark jobs, build Docker images, etc.
- Working knowledge of Data warehousing, Data modelling, Governance and Data Architecture
Good to have:
- Experience working with Data platforms, including EMR, Airflow, Data bricks (Data Engineering & Delta Lake components, and Lakehouse Medallion architecture), etc.
- Experience working in Agile and Scrum development process.
- Experience in EMR/ EC2, Data bricks etc.
- Experience working with Data warehousing tools, including SQL database, Presto, and Snowflake
- Experience architecting data product in Streaming, Server less and Microservices Architecture and platform.
Job Classification
Industry: IT Services & ConsultingFunctional Area:
Role Category: Software Development
Role: Software Development
Employement Type: Full time
Education
Under Graduation: B.Tech/B.E. in Any SpecializationPost Graduation: Any Postgraduate
Doctorate: Doctorate Not Required
Contact Details:
Company: NEXPLACE INFO PRIVATE LIMITEDLocation(s): Bengaluru
+ View Contact
Keyskills: Airflow data bricks SCALA AWS Python Jenkins Github databricks Spark EMR SQL
Fraud Alert to job seekers!
₹ 27.5-37.5 Lacs P.A
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in
