Python PySpark Data Engineer
- Infobeans
- 6 - 11 years
- Indore
- 2 months ago
- Email to a friend
- Report this job
Job Description
- Design, develop, and maintain robust data pipelines and applications using Python and PySpark.
- Build end-to-end data solutions from scratch, ensuring clean, modular, and reusable code.
- Write and optimize complex SQL queries for data transformation, validation, and analytics.
- Develop scalable, reusable frameworks following best practices in coding and architecture.
- Refactor existing codebases to improve performance and maintainability.
- Collaborate with data engineers, analysts, and stakeholders to deliver efficient and impactful solutions.
- Utilize Databricks and Azure Data Factory (ADF) to orchestrate and automate data workflows.
- Contribute to code reviews, performance tuning, and ensure adherence to development standards.
- Youll have the opportunity to build solutions from the ground up and shape scalable data products.
- Youll work hands-on with modern technologies like Databricks, ADF, and Azure.
- Youll collaborate with a talented, cross-functional team that values clean code and innovative thinking.
- Youll grow your expertise in data engineering best practices and cloud platforms.
- Youll be part of an environment that encourages continuous learning and ownership.
- 5+ years of hands-on experience in software or data engineering with a strong focus on Python and PySpark.
- Proven experience developing Python applications from scratch with modular and reusable code.
- Proficiency in SQL and experience working with large, complex datasets.
- Good understanding of data structures, design patterns, and scalable architecture.
- Hands-on experience with Databricks and Azure Data Factory (ADF).
- Knowledge of cloud platforms such as Azure, AWS, or GCP is a plus.
- Strong problem-solving skills with a solution-oriented and analytical mindset.
- A Bachelors degree in Computer Science, Engineering, or a related field.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: InfobeansLocation(s): Indore
Keyskills: python pyspark microsoft azure data engineering sql hive oracle sql queries performance tuning data validation cloud platforms azure data factory data bricks gcp spark design patterns oracle adf data structures data transformation hadoop sqoop aws
Similar positions
Infobeans
InfoBeans Systems is a fast growing software development company with offices in Pune and Indore and business development centre in US. We develop customized business applications for large and mid-size organizations in the US. We strongly believe that customer success defines our success. Hence, we...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in