Big Data Engineer
- Highbrow Technology
- 5 - 8 years
- Chennai
- 3 months ago
- Email to a friend
- Report this job
Job Description
We are looking for a skilled Big Data Developer with hands-on experience in Hadoop ecosystem, PySpark programming. The ideal candidate should be able to design, develop, and maintain data processing solutions using big data technologies.
Key Responsibilities:
- Develop and maintain scalable data pipelines using PySpark and Hadoop.
- Write efficient Shell scripts for automation and job orchestration.
- Process and transform large datasets in distributed environments.
- Work with Hive, HDFS, and other Hadoop tools for data handling.
- Collaborate with data engineers, analysts, and business teams to understand requirements.
- Monitor and optimize performance of data jobs.
Required Skills:
- Strong experience with Hadoop ecosystem (HDFS, Hive, etc.)
- Proficient in PySpark for big data processing.
- Good knowledge of Shell scripting in Unix/Linux environment.
- Experience with scheduling tools like Oozie or Airflow (optional but preferred).
- Understanding of distributed computing and performance tuning
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Big Data Engineer
Employement Type: Full time
Contact Details:
Company: Highbrow TechnologyLocation(s): Chennai
+ View Contact
Fraud Alert to job seekers!
₹ 2-6 Lacs P.A
Similar positions
Highbrow Technology
New Chennai ODC Address ASV Chandilya Towers,No-5/397,Rajiv Gandhi Salai,Okkiyam,Thoraipakkam,Chennai,600096.
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in