Senior Big Data Developer - Hadoop/ Spark
- Career Infosystem
- 5 - 10 years
- Noida, Gurugram
- 11 months ago
- Email to a friend
- Report this job
Job Description
Professional Summary :
- Senior Big Data Developer with complete expertise in executing Data Engineering and Data Analytics projects from scratch.
- Around 8 years of experience in Software Development with strong focus on Big Data, Hadoop and Java.
- Strong expertise in Big Data ecosystem like Spark, Hive, Sqoop, HDFS, Map Reduce, Kafka, Oozie, Yarn, Hbase, Nifi.
- Strong expertise in building scalable applications using various programming languages (Java, Scala and Python).
- In depth Knowledge on Architecture of Distributed Systems and Parallel Computing.
- Experience implementing end to end data pipelines for serving reporting and data science capabilities.
- Experience in fine tuning applications written in Spark and Hive and to improve the overall performance of the pipelines.
- Developed production ready Spark applications using Spark RDD apis, Data frames, Datasets, Spark SQL and Spark Streaming.
- Strong experience working with various configurations of Spark like broadcast thresholds, increasing shuffle partitions, caching, repartitioning etc., to improve the performance of the jobs.
- In depth knowledge on import/export of data from Databases using Sqoop.
- Well versed in writing complex hive queries using analytical functions.
- Knowledge in writing custom UDF's in Hive to support custom business requirements.
- Solid experience in using the various file formats like CSV, TSV, Parquet, ORC, JSON and AVRO.
- Experience in using the compression techniques like Gzip, Snappy with in Hadoop.
- Strong knowledge of NoSQL databases and worked with HBase, Cassandra and Mongo DB.
- Experience in using the cloud services like Amazon EMR, S3, EC2, Red shift and Athena.
- Experience automating end-to-end data pipelines with strong resilience and recoverability.
- Worked on Spark Streaming and Structured Spark streaming including Kafka for real time data processing.
- Strong knowledge of version control systems like SVN and GITHUB.
- Involved in production monitoring using workflow monitor and experience in development and support environments.
- Experienced in using waterfall, Agile and Scrum models of software development process framework.
- Good level of experience in Core Java, J2EE technologies, JDBC, Servlets and JSP.
- Good knowledge in Oracle PL/SQL and shell scripting.
- Experience in design and development of Web forms using Spring MVC, Java Script, JSON and JQ plotter.
- Active team player with excellent interpersonal skills, keen learner with self-commitment& innovation.
- Ability to meet deadlines and handle pressure in coordinating multiple tasks in the work environment.
Technical Skills :
- Big Data Ecosystem: Spark, MapReduce, HDFS, HIVE, HBase, Pig, Sqoop, Flume, Oozie, Zookeeper, Spark, Hue, Cloudera (CDH), Hortonworks(HDP)
- Cloud Services: EC2, EMR, S3, Redshift, Athena, AWS ECS, Terraform, AWS Cloud Formation, AWS Cloud Watch,
- Relational Databases: Oracle 12c, MySQL, MS-SQL Server
- NoSQL Databases: HBase, Cassandra and MongoDB
- Version Control: GitHub, SVN
- Programming Languages: Java, Scala, Python, SQL, PL/SQL, Shell Scripting.
- Software Development: UML, Design Patterns (Core Java and J2EE), Software Development Lifecycle (SDLC), Waterfall Model and Agile, STLC
- Web Technologies: JavaScript, CSS, HTML and JSP.
- Operating Systems: Windows, UNIX/Linux, and Mac OS.
- Build Management Tools: Maven, Ant.
- IDE & Command line tools: Eclipse, IntelliJ, Toad and NetBeans.
Job Classification
Industry: IT Services & ConsultingFunctional Area:
Role Category: Software Development
Role: Software Development
Employement Type: Full time
Education
Under Graduation: B.Tech/B.E. in Any Specialization, B.Sc in Any Specialization, BCA in ComputersPost Graduation: Any Postgraduate
Doctorate: Doctorate Not Required
Contact Details:
Company: Career InfosystemLocation(s): Noida, Gurugram
Keyskills: Java Hadoop Big Data HDFS Kafka J2EE Spark Data Analytics Distributed Systems MapReduce HBase
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in
