Data Engineer(Spark/Scala/Cloudera)
- Photon
- 6 - 9 years
- Pune
- 15 days ago
- Email to a friend
- Report this job
Job Description
Job Title: Data Engineer (Spark/Scala/Cloudera)
Location: Chennai/Pune
Job Type: Full time
Experience Level: 6- 9 years
Job Summary:
We are seeking a skilled and motivated Data Engineer to join our data engineering team. The ideal candidate will have deep experience with Apache Spark, Scala, and Cloudera Hadoop ecosystem. You will be responsible for building scalable data pipelines, optimizing data processing workflows, and ensuring the reliability and performance of our big data platform.
Key Responsibilities:
Design, build, and maintain scalable and efficient ETL/ELT pipelines using Spark and Scala.
Work with large-scale datasets on the Cloudera Data Platform (CDP).
Collaborate with data scientists, analysts, and other stakeholders to ensure data availability and quality.
Optimize Spark jobs for performance and resource utilization.
Implement and maintain data governance, security, and compliance standards.
Monitor and troubleshoot data pipeline failures and ensure high data reliability.
Participate in code reviews, testing, and deployment activities.
Document architecture, processes, and best practices.
Required Skills and Qualifications:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field.
- 6+ years of experience in big data engineering roles.
- 2 + Years of Hands on experience into Scala
- Proficient in Apache Spark (Core/DataFrame/SQL/RDD APIs).
- Strong programming skills in Scala.
- Hands-on experience with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Oozie).
- Familiarity with distributed computing and data partitioning concepts.
- Strong understanding of data structures, algorithms, and software engineering principles.
- Experience with CI/CD pipelines and version control systems (e.g., Git).
- Familiarity with cloud platforms (AWS, Azure, or GCP) is a plus.
Preferred Qualifications:
- Experience with Cloudera Manager and Cloudera Navigator.
- Exposure to Kafka, NiFi, or Airflow.
- Familiarity with data lake, data warehouse, and lakehouse architectures.
Preferred candidate profile
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Data Engineer
Employement Type: Full time
Contact Details:
Company: PhotonLocation(s): Pune
Keyskills: Data Engineering Scala Programming Cloudera Spark Programming Big Data
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in