Data Engineer @ Incedo

Roles & Responsibilities:
Designing, developing and maintaining large-scale big data platforms using technologies like Hadoop, Spark and Kafka
Creating and managing data warehouses, data lakes and data marts
Implementing and optimizing ETL processes and data pipelines
Developing and maintaining security and access controls
Troubleshooting and resolving big data platform issues
Collaborating with other teams to ensure the consistency and integrity of data

Skills Requirements:

• Experience with big data processing technologies such as Apache Hadoop, Apache Spark, or Apache Kafka.
• Understanding of distributed computing concepts such as MapReduce, Spark RDDs, or Apache Flink data streams.
• Familiarity with big data storage solutions such as HDFS, Amazon S3, or Azure Data Lake Storage.
• Knowledge of big data processing frameworks such as Apache Hive, Apache Pig, or Apache Impala.
• Must have excellent communication skills and be able to communicate complex technical information to non-technical stakeholders in a clear and concise manner.
• Must understand the company's long-term vision and align with it.
• Provide leadership, guidance, and support to team members, ensuring the successful completion of tasks, and promoting a positive work environment that fosters collaboration and productivity, taking responsibility of the whole team.

Qualifications

• 4-6 years of work experience in relevant field
• B.Tech/B.E/M.Tech or MCA degree from a reputed university. Computer science background is preferred