Python Pyspark Data Engineer @ Infobeans

Overview:

We are seeking a highly skilled Python and PySpark Developer who is passionate about building robust data solutions from the ground up. The ideal candidate should have a deep understanding of core programming concepts, be highly proficient in Python and PySpark, and possess strong SQL skills. We're looking for someone who thinks out of the box, understands the value of reusable and scalable solutions, and is comfortable working in dynamic, fast-paced environments. Experience with Databricks and Azure Data Factory (ADF) is needed.

Responsibilities:

• Design, develop, and maintain Python and PySpark.

• Write complex SQL queries for data transformation, validation, and analysis.

• Build scalable, reusable code frameworks with a dynamic and modular mindset.

• Develop solutions from scratch with clean, well-documented code.

• Optimize and refactor existing code for better performance and scalability.

• Collaborate closely with data engineers, analysts, and business stakeholders to deliver impactful solutions.

• Implement best practices in data engineering, coding standards, and deployment.

• Work with Databricks and ADF to orchestrate and manage data workflows.

Requirements:

• Strong foundational knowledge and hands-on experience in Python and PySpark.

• Should have experience building applications on Python from scratch

• Proficiency in SQL and experience working with large datasets.

• Demonstrated ability to code from scratch, not just maintain or tweak existing codebases.

• Ability to think innovatively and approach problems with a solution-oriented mindset.

• Understanding of reusable design patterns and dynamic solutioning.

• Experience working with Databricks and Azure Data Factory (ADF).

• Knowledge of cloud data platforms (Azure, AWS, or GCP) is a plus.

• Bachelor's degree in Computer Science, Engineering, or a related field.

+ View Contact

Keyskills:   Pyspark Databricks Python Data Factory