Big Data Engineer Lead @ Brillio

Home > Data Science & Machine Learning

Big Data Engineer Lead

Brillio
6 - 10 years
Bengaluru
4 months ago
Email to a friend
Report this job

Job Description

Key Skills - Hadoop, PySpark, Python, SQL, Apache Iceberg (niche) , Hive

Requirements

Solid knowledge of Hadoop ecosystem such as Hive, Iceberg, Spark SQL and Proficiency in Python, Unix ,SQL
Experience with Apache Kafka, Apache Flink, and other relevant streaming technologies.
Strong hands-on experience with Hive and SQL for querying and data transformation
Proficiency in Python for data manipulation and automation
Expertise in Apache Spark (batch and streaming)
Deep understanding of Hadoop ecosystem (HDFS, YARN, MapReduce)
Experience working with Kafka for streaming data pipelines
Experience with workflow orchestration tools (Airflow, Oozie, etc.)
Knowledge of cloud-based big data platforms (AWS EMR, GCP Dataproc, Azure HDInsight)
Fam
iliarity with CI/CD pipelines and version control (Git)

Responsibilities

Design, develop, and maintain scalable data pipelines and ETL processes.
Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
Write efficient, high-performance SQL queries to extract, transform, and load data.
Develop reusable data frameworks and utilities in Python.
Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.

Job Classification

Industry: IT Services & Consulting
Functional Area / Department: Data Science & Analytics
Role Category: Data Science & Machine Learning
Role: Data Engineer
Employement Type: Full time

Contact Details:

Company: Brillio
Location(s): Bengaluru

+ View Contact

Login

Candidates can login here to view contacts and apply.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach Resume Max 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Candidates are expected to provide most recent and accurate profile information, inappropriate content is strictly prohibited!

Keyskills: Pyspark Hive Iceberg Hadoop Python SQL

Job seems aged, it may have been expired!
Fraud Alert to job seekers!

₹ Not Disclosed

Job application

We will notify the employer with your details. You can also attach a resume or a cover letter.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

Expected Salary (*)

Desired Industry (*):

Functional area / Department (*):

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach ResumeMax 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Similar positions

Senior Data Scientist

Tredence

5 - 9 years

Pune

1 day ago

₹ Not Disclosed

Data Scientist

Tata Consultancy

10 - 18 years

Bengaluru

2 days ago

₹ Not Disclosed

Data Scientist (PhD in Statistics / AI)

Tata Consultancy

10 - 18 years

Bengaluru

2 days ago

₹ Not Disclosed

Principal, Data Scientist

Walmart

3 - 5 years

Bengaluru

2 days ago

₹ Not Disclosed

Brillio

Powered by SophosLabs a global threat intelligence and data science team Sophoscloud-native and AI-enhanced solutions secure endpoints (laptops, servers and mobile devices) and networks against evolving cybercriminal tactics and techniques, including automated and active-adversary breaches, ransomwa...

Big Data Engineer Lead @ Brillio

Home > Data Science & Machine Learning