Level 3 Big Data Developer
- Celcom Solutions
- 4 - 6 years
- Bengaluru
- 7 months ago
- Email to a friend
- Report this job
Job Description
- Should be able to design, develop, and implement highly scalable and distributed big data solutions using Hadoop ecosystem technologies such as HBase, Hive, Kudu, and Spark.
- Should have knowledge on Architect HBase schemas and data models to accommodate evolving business requirements and ensure optimal performance for data storage and retrieval operations.
- Should have knowledge on developing complex Hive queries and data processing pipelines to transform raw data into structured formats suitable for analysis and reporting.
- Implement data ingestion pipelines using Spark Streaming and Spark SQL for real-time processing of streaming data sources, ensuring high throughput and low latency.
- Optimize Spark applications for performance and resource utilization, including tuning RDD transformations, optimizing data partitioning strategies, and leveraging in-memory caching.
- Utilize advanced features of Spark MLlib for machine learning tasks such as classification, regression, clustering, and collaborative filtering.
- Design and deploy Kudu tables for fast analytical queries and real-time analytics, leveraging Kudu s unique combination of fast analytics and fast data ingestion.
- Collaborate with data scientists to integrate machine learning models into Spark workflows and productionize them for real-time predictions and analytics.
- Troubleshoot performance bottlenecks, data quality issues, and system failures in big data applications and infrastructure, and implement solutions to address them.
- Stay abreast of emerging technologies and best practices in big data processing and analytics, and evaluate their potential impact on our architecture and solutions.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Software Development - Other
Employement Type: Full time
Contact Details:
Company: Celcom SolutionsLocation(s): Bengaluru
Keyskills: Resource utilization spark Analytical Machine learning Data processing Data quality big data Analytics SQL HBase
Similar positions
Celcom Solutions
Celcom Solutions was founded in Reading UK, in 2010. In 2012, we expanded our business to Chennai and Bangalore, India in 2012 and in Atlanta, US in 2015. Celcom Solutions has a combined professional experience of more than 20 years. The consultants at Celcom Solutions have had extensive hands on ex...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in