Senior Data Engineer
- Quantum Integrators
- 8 - 13 years
- Hyderabad
- 11 days ago
- Email to a friend
- Report this job
Job Description
Role: Senior Data Engineer
Location: Hyderabad, TS. Hybrid
Type of job: Full Time
Working Hours: UK/US
Joining Timeline: Within 30 days of offer confirmation (immediate joiners preferred)
The Senior Data Engineer will be responsible for creating a connected data ecosystem that unleashes the power of our streaming data. We gather data, to allow fast feedback loops across territories; combined with our strategic data platform, this data ecosystem is at the core of being able to make intelligent business decisions.
In this role, the Senior Data Engineer will share responsibilities in the development and maintenance of data pipelines that facilitate data driven machine learning, as well as support ongoing operations related to the Life Sciences and Pharmaceuticals
Responsibilities include, but are not limited to:
- Collaborate with a high-performance team of Data Engineers
- Contribute to and facilitate design, build, testing, scaling, and maintaining data pipelines from a variety of source systems and (Internal, third party, cloud based, etc.), according to business and technical requirements.
- Deliver observable, reliable and secure software, embracing you build it you run it mentality, and focus on automation and GitOps.
- Continually work on improving the codebase and have active participation and oversight in all aspects of the team, including agile ceremonies.
- Take an active role in story definition, assisting business stakeholders with acceptance criteria.
- Work with Principal Engineers and leaders to share and contribute to the broader technical vision.
- Develop and champion best practices, striving towards excellence and raising the bar within the department.
- Develop solutions combining data blending, profiling, mining, statistical analysis, and machine learning, to better define and curate models, test hypothesis, and deliver key insights
- Operationalize data processing systems
Qualifications
- 6+ years relevant experience in Data Engineering
- Programming skills in one or more of the following: Python, Java, Scala
- Proven ability to write reusable/efficient code to automate analysis and data processes
- Demonstrated expertise in Advanced SQL, ETL, Data Architecture, and Workflow management
- Intermediate level of experience querying relational databases with SQL
- Demonstrated experience implementing scalable, distributed, and highly available systems in a cloud ecosystem
- Ability to work with graph-based data workflows (e.g., Apache Airflow), or any ETL / ELTs
- Expertise in crafting advanced SQL queries and optimizing database performance.
- Basic data modeling, transforming data and deriving strategic insights.
- Familiarity with Cloud Datawarehouses (Snowflake, Databricks, Big Query) for processing and analyzing large datasets.
- Extensive experience in ETL processes for seamless data movement across systems.
- Ability to develop data catalogs while ensuring clarity and correctness of key business metrics
- Experience building streaming data pipelines using Kafka, Spark or Flink
- Data modelling experience (operationalizing data science models/products a plus)
- Cloud certifications (Azure, GCP, AWS)
- Bachelors degree with a specialization in Computer Science, Engineering, Physics, other quantitative fields or equivalent industry experience.
Desired Characteristics
- Experience in processing structured and unstructured data into a form suitable for analysis and reporting with integration with a variety of data metric providers ranging from advertising, web analytics, and consumer devices
- Experience building and deploying ML pipelines: training models, feature development, regression testing
- Strong Test-Driven Development background, with understanding of levels of testing required to continuously deliver value to production.
- Experience with large-scale data for analytics (media experience a plus)
- Ability to work effectively across functions, disciplines, and levels
- Team-oriented and collaborative approach with a demonstrated aptitude, enthusiasm and willingness to learn new methods, tools, practices and skills
- Ability to surface and navigate differing perspectives constructively to drive alignment and sound technical decisions
- Pride and ownership in your work and confident representation of your team to others
- Desired Certification one of the cloud platforms or Datawarehouse platforms
Additional Requirements
- Hybrid: This position currently has a hybrid schedule, which requires contributing from the office a minimum of four days per week. The Company reserves the right to change in-office requirements at any time.
Regards,
Jagannath Gaddam
ja*************m@qu***************s.com
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Software Development - OtherEmployement Type: Full timeContact Details:
Company: Quantum IntegratorsLocation(s): Hyderabad
Keyskills: Data Engineering SQL Python Java Big Query graph-based data workflows Scala Workflow management Snowflake Data Architecture Databricks ETL Advanced SQL
Similar positions
Quantum Integrators
\\\\\\\\n\\\\\\\\n Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success, we have evolved to become on...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in