Job Description
Responsibilities :
- Responsible to Ingest data from files, streams and databases. Process the data with Python and Pyspark and its storage into time series database.
- Develop programs in Python as part of data extraction, data cleaning, transformation and processing.
- Develop and maintain scalable data pipelines.
- Rest API's development.
Mandatory Skills :
- Overall 6 -8 years of experience as Data Engineer.
- Advanced working SQL knowledge to create complex queries.
- Experience in working with Time-series Database, relational databases as well as working familiarity with a variety of databases (structured & Unstructured).
- Experience in designing and implementing scalable architecture.
- Good experience in doing object-oriented programming in python.
- Very strong in Object Oriented Analysis and Design (OOAD).
- Strong knowledge on REST APIs.
- Experience working on Azure Cloud services IaaS, PaaS.
- Hands on Experience in working on Microsoft Azure Services like ADLS/Blob Storage solutions, Event Hubs, Service Bus, scale sets, Load Balancers, Azure Functions, Databricks.
- Hands on Experience in working on Kafka.
- Knowledge on continuous integration/continuous deployment.
- Experience of data migration and deployment from On-Prem to Cloud environment and vice-versa.
- Individual Contributor.
Nice-to-Have Skills :
- Hands on experience on visualization tools like Grafana & Power BI.
- Certification on any cloud platform.
- Hands on experience on Docker Containers and micro services architecture using Kubernetes.
- Hand on experience in Golang programming.
Employement Category:
Employement Type: Full time
Industry: IT
Functional Area: IT
Role Category: Software Engineer
Role/Responsibilies: Immediate Joinees Required for Lead Data Engineer with Fintech Start-up in Bangalore
Contact Details:
Company: Cerentral Consultants
Location(s): Bengaluru
Keyskills:
Azure
BIGDATA
PYTHON
Pyspark
Apache