Atleast 2 years of Lead experience is required to apply for this role.
Your JD
Design and build reusable components, frameworks and libraries at scale to support analytics products.
Design and implement product features in collaboration with business and Technology stakeholders.
Anticipate, identify and solve issues concerning data management to improve data quality.
Clean, prepare and optimize data at scale for ingestion and consumption.
Drive the implementation of new data management projects and re-structure of the current data architecture.
Implement complex automated workflows and routines using workflow scheduling tools.
Build continuous integration, test-driven development and production deployment frameworks.
Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards.
Analyze and profile data for the purpose of designing scalable solutions.
Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues.
Mentor and develop other data engineers in adopting best practices.
Qualifications:Primary skillset:
Experience working with distributed technology tools for developing Batch and Streaming pipelines using.
SQL, Spark, Python [6+ years],
Airflow [4+ years],
Scala [4+ years].
Experience in Cloud Computing, e.g., AWS, GCP, Azure, etc.
Able to quickly pick up new programming languages, technologies, and frameworks.
Strong skills building positive relationships across Product and Engineering.
Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
Experience with creating/ configuring Jenkins pipeline for smooth CI/CD process for Managed Spark jobs, build Docker images, etc.
Working knowledge of Data warehousing, Data modelling, Governance and Data Architecture
Good to have:
Experience working with Data platforms, including EMR, Airflow, Data bricks (Data Engineering & Delta Lake components, and Lakehouse Medallion architecture), etc.
Experience working in Agile and Scrum development process.
Experience in EMR/ EC2, Data bricks etc.
Experience working with Data warehousing tools, including SQL database, Presto, and Snowflake
Experience architecting data product in Streaming, Server less and Microservices Architecture and platform.
Job Classification
Industry: IT Services & Consulting Functional Area: Role Category: Software Development Role: Software Development Employement Type: Full time
Education
Under Graduation: B.Tech/B.E. in Any Specialization Post Graduation: Any Postgraduate Doctorate: Doctorate Not Required
Contact Details:
Company: NEXPLACE INFO PRIVATE LIMITED Location(s): Bengaluru