Lead Software Engineer - Bengaluru/Bangalore
- CPP Investments
- 2 - 10 years
- Bengaluru
- 1 month ago
- Email to a friend
- Report this job
Job Description
- Lead, design and code solutions using Big Data/Hadoop/database for ensuring application access to enable data driven decision making for the company s multi-faceted ad serving operations.
- Working closely with Engineering resources across the globe to ensure enterprise data solutions and assets are actionable, accessible and evolving in lockstep with the needs of the ever-changing business model.
- Ideal candidate can lead in the areas of: solution design, code development, quality assurance, data processing, cross team communication, project management, and application maintenance.
- You have hands on experience in - Kafka, Flume, Spark, Java/Scala, Hadoop, HDFS, Hive,SQL
- to work with Epsilon Market place team
- You have hands on experience in coding languages like Python & Scala , fine tuning Spark jobs.
- You have exposure on Airflow, Docker container, noSQL databses like Hbase.
What you will enjoy in this role
- As part of Data Pipeline team will be processing Billions of data per day from multiple region/ data center s.
- Working on processing Ad-server data into Storage layer where further analytics will be done
- Working on Bigdata technologies like Flume, Kafka, Spark, and loads the aggregated/processed data into HDFS.
- Working on identifying a key area of ownership the team has, pipelining data.
- Working on intraday (hourly, 5 minute ,15 minute) aggregations using Spark Structured Streaming to jobs performed.
- Working on the data assets which will be further used in performance measurement and efficacy of the defined solution, as well as feeding business analytics and data mining.
What you will do
- Take E2E ownership & lead the Team by delivering product on-time with high quality
- End to end development of automated receipt of anonymized data
- End to end development of processing of logs data
- Data center to data center replication
- Data processing using Flume, Kafka , Spark jobs, Airflow,Docker etc
- Migration of production data assets to downstream consuming systems
- Disaster Recovery and Business Continuity implementations
- Workload Performance Management and Tuning
- Ensuring that coded solutions meet functional business requirements for ad serving and measurement
- Application specific controls and scheduling
- Custom solution building for syndicated and third party datasets
- Bachelor s Degree in Computer Science or equivalent degree is required.
- 7 - 12 years of data engineering experience around database marketing technologies and data management, and technical understanding in these areas.
- Strong hands on experience in opensource components like Kafka, Flume, Spark, Hadoop, HDFS, Hive, Java/Scala, SQL, Experience with Scripting - Python preferred
- Minimum of 2 years of Lead experience in running teams.
- Ability to handle complex products
- Strong understanding of Data structures and algorithms.
- Good experience in working with geographically and culturally diverse teams.
- Strong understanding of Disaster Recovery and Business Continuity solutions
- Experience with scheduling applications like Airflow, Oozie with complex interdependencies
- Experience with Docker ,Kubernetes a plus
- Strong understanding of Disaster Recovery and Business Continuity solutions
- Familiarity with complex data lake environments that span OLTP, MPP and Hadoop platforms
- Ability to diagnose and troubleshoot problems quickly
- Good experience in working with geographically and culturally diverse teams
- Excellent written and verbal communication skills.
Job Classification
Industry: BankingFunctional Area: Banking
Role Category: DBA / Data warehousing
Role: Database Architect / Designer
Employement Type: Full time
Contact Details:
Company: CPP InvestmentsLocation(s): Bengaluru
+ View Contact
Keyskills: Manager Quality Assurance Data management Performance management Coding Project management Business analytics Data processing Scheduling Data mining SQL
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
CPP Investments
About companyCPP Investments is a professional investment management organization with a vital purpose: to help provide a foundation on which Canadians build financial security in retirement. To help maximize the CPP Fund for generations to come, we invest across all major asset classes: public equi...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in
