Lead Data Engineer
- Incedo
- 7 - 9 years
- Hyderabad
- 3 months ago
- Email to a friend
- Report this job
Job Description
As a Technical Lead - Big Data Platform at Incedo, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects. You will work with data architects and data engineers to define the platform architecture and build the necessary infrastructure. You will be skilled in big data technologies such as Hadoop, Spark, and Kafka and have experience in cloud computing platforms such as AWS or Azure. You will be responsible for ensuring the performance, scalability, and security of the big data platform and troubleshooting any issues that arise.
Roles & Responsibilities:
- Designing, developing and maintaining large-scale big data platforms using technologies like Hadoop, Spark and Kafka
- Creating and managing data warehouses, data lakes and data marts Implementing and optimizing ETL processes and data pipelines
- Developing and maintaining security and access controls Troubleshooting and resolving big data platform issues Collaborating with other teams to ensure the consistency and integrity of data
Skills Requirements:
- Experience with big data processing technologies such as Apache Hadoop, Apache Spark, or Apache Kafka. Understanding of distributed computing concepts such as MapReduce, Spark RDDs, or Apache Flink data streams.
- Familiarity with big data storage solutions such as HDFS, Amazon S3, or Azure Data Lake Storage. Knowledge of big data processing frameworks such as Apache Hive, Apache Pig, or Apache Impala.
- Must have excellent communication skills and be able to communicate complex technical information to non-technical stakeholders in a clear and concise manner.
- Must understand the company's long-term vision and align with it. Should be open to new ideas and be willing to learn and develop new skills. Should also be able to work well under pressure and manage multiple tasks and priorities.
Qualifications
- 7-9 years of work experience in relevant field B.Tech/B.E/M.Tech or MCA degree from a reputed university. Computer science background is preferred
Job Classification
Industry: BankingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: IncedoLocation(s): Hyderabad
Keyskills: hive apache pig impala spark hadoop azure data lake data processing big data technologies microsoft azure apache flink data engineering data mart apache mapreduce computer science kafka aws etl data integration data lake etl process
Similar positions
Incedo
If youre thinking scale, think bigger and dont stop there. At Walmart Global Tech India, we dont just innovate, we enable transformations across stores and different channels for the Walmart experience. \\\\\\\\r\\\\\\\\n \\\\\\\\r\\\\\\\\nA regular day at Walmart Global Tech India means using tech...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in