Assistant Manager - Spark Developer
- Genpact India
- 4 to 6 Yrs
- Hyderabad
- 3 years ago
- Email to a friend
- Report this job
Job Description
* With a startup spirit and 90,000+ curious and courageous minds, we have the expertise to go deep with the world s biggest brands and we have fun doing it. Now, we re calling all you rule-breakers and risk-takers who see the world differently, and are bold enough to reinvent it. Come, transform with us. Are you the one we are looking for We are inviting applications for the role of AM, Spark DeveloperResponsibilities- Should have experience working on Spark and SQL modules of Spark extensively
- Experience in designing and developing applications in Spark using Scala to compare the performance of Spark with Hive and SQL/Oracle.
- Should have experience in analyzing Hive SQL scripts and crafted a solution to implement using Scala
- Should have experience in crafting and developing applications in Spark using Scala to compare the performance of Spark with Hive and SQL/Oracle.
- Develop Spark scripts by using Scala shell commands as per the requirement
- Develop Scala scripts, UDFs using both Data frames/SQL/Data sets and RDD/MapReduce in Spark for Data Aggregation, queries and writing data back into OLTP system through Sqoop
- Optimizing of existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames and Pair RDD's.
- Experienced in performance tuning of Spark Applications for setting right Batch Interval time, accurate level of Parallelism and memory tuning
- Expertise with the tools in Hadoop Ecosystem including Hive, HDFS, MapReduce, Sqoop, Spark, Kafka, Yarn
- Excellent knowledge on Hadoop ecosystems such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node
- Very good understanding of Partitions, Bucketing concepts in Hive and crafted both Managed and External tables in Hive to optimize performance.
- Experienced in handling large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformations and other during ingestion process itself
- Develop Hive queries to process the data and generate the data cubes for visualizing
- Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
- Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, and Scala.
- Experience in manipulating/analyzing large datasets and finding patterns and insights within structured and unstructured data
- Should be involved in building Hive tables, and loading and analyzing data using hive queries
- Proven experience with SQL queries and database tuning
- Strong knowledge of database design and development with previous experience in developing ETL processes, and multifaceted data models
- Respond to & solving support enquiries from users across various groups including Finance, Digital and Operations
- Programming languages: Java, C, C++, Scala, Python
- Scripting: Shell
- Operating systems: Linux, Unix, Windows
- RDBMS/No SQL: SQL Server, MySQL, Oracle 11g, Azure, HBase
- Hadoop Ecosystem: Spark Scala, Spark Python, Map Reduce, Hive, HBase, HDFS, Sqoop, Pig, , Zookeeper, Kafka, Spark streaming, Oozie
- Analytical thinking and problem solving abilities.
- Good Presentation Skills
Employement Category:
Employement Type: Full timeIndustry: BPO / Call Center
Role Category: DBA / Datawarehousing
Functional Area: Not Applicable
Role/Responsibilies: Assistant Manager - Spark Developer
Contact Details:
Company: Genpact IndiaLocation(s): Hyderabad
+ View Contact
Fraud Alert to job seekers!
₹ Not Specified
Similar positions
Genpact India
Genpact (NYSE: G) is a global professional services firm focused on delivering digital transformation for our clients, putting digital and data to work to create competitive advantage. We do this by integrating lean principles, design thinking, analytics and digital technologies with our domain and ...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in
