Data Engineer
- o9 Solutions
- 4 - 8 years
- Bengaluru
- 2 months ago
- Email to a friend
- Report this job
Job Description
- Work on the data pipelines for capturing historical snapshots of both inputs and product outputs
- Performance tuning of the pipelines using industry best practices
- Batch orchestration design and development in a way that is least disruptive to system usage
- Develop PySpark/Python codes for data transformation and API data extraction in batch jobs
- Contribute to overall product architecture and make it best-in-class from a performance and scalability standpoint
- Learn new technologies as needed for product usecases
What should you have
- 4 years of coding experience in Pyspark
- Worked with different aspects of the Spark ecosystem, including Spark SQL, DataFrames, Datasets, and streaming data
- 4+ years of experience as a data engineer
- Deep understanding and experience in Pyspark and some experience in the data lake and delta tables
- Skilled in big data tools, building data pipelines, ETL design, and implementation
- Must have strong programming skills in Python Scala is a plus
- Should be familiar with Python (especially libraries like Pandas) The candidate should perform performance tuning and use Pyspark to move data
- Experienced writing production-level code, optimizing data processing, identifying performance bottlenecks, and root causes, and resolving defects
- Collaborates effectively with cross-functional teams to achieve product goals
- Familiar with software development best practices (Git, CI/CD, Unit Testing)
Job Classification
Industry: Film / Music / EntertainmentFunctional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Data Engineer
Employement Type: Full time
Contact Details:
Company: o9 SolutionsLocation(s): Bengaluru
+ View Contact
Keyskills: Performance tuning GIT orchestration Coding SCALA Data processing Unit testing SQL Python Data extraction
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
o9 Solutions
We help improve and integrate all core planning processes of an organization in one smart.simple.fun management system.
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in