Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Big Data Architect (Spark, Kafka and Python) @ Crown Solutions

Home > System Design / Implementation / ERP / CRM

 Big Data Architect (Spark, Kafka and Python)

Job Description

Dear Candidate,


Hope you're doing well!!!!


We are looking for an experienced Big Data with Spark, Kafka and Python.


Position: Big Data Architect

Experience: 9+ Yrs to 15+ Yrs

Location: Bangalore


Skills & Experience


Skills and attributes for success

  • Excellent communicator (written and verbal formal and informal).
  • Ability to multi-task under pressure and work independently with minimal supervision.
  • Strong verbal and written communication skills.
  • Must be a team player and enjoy working in a cooperative and collaborative team environment.
  • Adaptable to new technologies and standards.

Technology specific (Viz/ETL/MDM/Bigdata etc)

  • Strong understanding & familiarity with all Hadoop Ecosystem components and Hadoop Administrative Fundamentals
  • Strong understanding of underlying Hadoop Architectural concepts and distributed computing paradigms
  • Experience in the development of Hadoop APIs and MapReduce jobs for large scale data processing.
  • Experience in architecting big data solutions with proven track record in driving business success
  • Hands-on programming experience in Apache Spark using SparkSQL and Spark Streaming or Apache Storm
  • Hands on experience with major components like Hive, PIG, Spark, MapReduce
  • Experience working with NoSQL in at least one of the data stores - HBase, Cassandra, MongoDB
  • Experienced in Hadoop clustering and Auto scaling.
  • Good knowledge in apache Kafka & Apache Flume
  • Knowledge of Spark and Kafka integration with multiple Spark jobs to consume messages from multiple Kafka partitions
  • Knowledge of Apache Oozie based workflow
  • Hands-on expertise in cloud services like AWS, or Microsoft Azure
  • Experience with databricks, glue, python, step functions or ADF
  • Solid understanding of ETL methodologies in a multi-tiered stack, integrating with Big Data systems like Hadoop and Cassandra.
  • Experience with BI, and data analytics databases
  • Experience in converting business problems/challenges to technical solutions considering security, performance, scalability etc.
  • Experience in Enterprise grade solution implementations.
  • Knowledge in Big data architecture patterns [Lambda, Kappa]
  • Experience in performance bench marking enterprise applications
  • Experience in Data security [on the move, at rest] and knowledge of data standards like APRA, BASEL etc
  • Develop standardized practices for delivering new products and capabilities using Big Data technologies, including data acquisition, transformation, and analysis.
  • Define and develop client specific best practices around data management within a Hadoop environment on Azure cloud
  • Recommend design alternatives for data ingestion, processing and provisioning layers
  • Design and develop data ingestion programs to process large data sets in Batch mode using HIVE, Pig and Sqoop technologies
  • Develop data ingestion programs to ingest real-time data from LIVE sources using Apache Kafka, Spark Streaming and related technologies
  • Strong UNIX operating system concepts and shell scripting knowledge
  • Knowledge of microservices and API development

To qualify for the role, you must have

  • BE/BTech/MCA/MBA with adequate industry experience
  • Minimum 8 years hand-on experience in one or more of the above areas.
  • Minimum 12 years industry experience

Ideally, youll also have
Project management skills
Familiar with Agile project delivery.
Exposure to cloud technologies such as Azure, AWS is preferred.

If you are interested, kindly forward your latest CV ka*****k@cr**********n.com

immediately incorporating the following details:


  • Current CTC:
  • Expected CTC
  • Notice period:

Also, it would be greatly appreciated if you refer any of your friends having experience in the same field


Job Classification

Industry: IT-Software, Software Services
Functional Area: IT Software - Application Programming, Maintenance,
Role Category: System Design/Implementation/ERP/CRM
Role: System Design/Implementation/ERP/CRM
Employement Type: Full time

Education

Under Graduation: Any Graduate in Any Specialization
Post Graduation: PG Diploma in Computers, MCA in Computers, MS/M.Sc(Science) in Computers, M.Tech in Computers
Doctorate: M.Phil. in Computers, Ph.D in Computers

Contact Details:

Company: Crown Solutions India (P) Ltd
Address: No - 40, 5th Street, Shakthi Nagar, Thoraipakkam, , Chennai, Tamilnadu, India
Location(s): Trivandrum

+ View Contactajax loader


Keyskills:   Apache Storm Hive Sqoop Hadoop Kafka Spark Oozie Apache Flume HBase Pig

 Job seems aged, it may have been expired!
 Fraud Alert to job seekers!

₹ Not Disclosed

Crown Solutions

If it is not success, then it is nothing else. No one will approach a failure but they will only ask about success stories. Therefore we work hard day in and day out just for this, success.Crown, today is a well renown name in the arena of global IT solutions & also as a service company providing te...