DataBricks/PySpark Developer
- JRD Systems
- 7 - 12 years
- Hyderabad
- 24 days ago
- Email to a friend
- Report this job
Job Description
Databricks/PySpark offshore Developer
Looking for an offshore Lead Databricks/PySpark Developer who is willing to learn new technologies if needed and able to work with a team.
Essential Job Functions:
- Design and development of data ingestion pipelines (Databricks background preferred).
- Performance tune and optimize the data bricks jobs
- Evaluated new features and refractors existing code
- Mentor junior developers and makes sure all patterns are documented
- Perform data migration and conversion activities.
- Develop and integrate software applications using suitable development methodologies and standards, applying standard architectural patterns, taking into account critical performance characteristics and security measures.
- Collaborate with Business Analysts, Architects and Senior Developers to establish the physical application framework (e.g. Libraries, modules, execution environments).
- Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform.
- Maintain and support the application.
- Must be willing to flex work hours accordingly to support application launches and manage production outages if necessary
- Ensures to understand the requirements thoroughly and in detail and identify gaps in requirements
- Ensures that detailed unit testing is done, handles negative scenarios and document the same
- Work with QA and automation team.
- Works on best practices and documenting the process
- code merges and releases (Bitbucket)
- Works with architect and manager on designs and best practices
- Good data analysis skills
Other Responsibilities:
* Safeguard the companys assets.
* Adhere to the companys compliance program.
* Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices.
* Maintain a focus on customer-service, efficiency, quality, and growth.
* Collaborating with additional team members
* Other duties as assigned.
Minimum Qualifications and Job Requirements:
* Must be a team player.
* Must have following
SCALA
SQL
Spark/Spark Streaming
Big Data Tool Set
Linux
Python/PySpark
Kafka
* Experience collaborating with dev team, project managers, and engineers.
* Excellent communication and teamwork skills
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & Analytics
Role Category: Data Science & Machine Learning
Role: Data Engineer
Employement Type: Full time
Contact Details:
Company: JRD SystemsLocation(s): Hyderabad
Keyskills: DataBricks Pyspark Python SQL c# Azure Databricks PLSQL
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in