A Data Engineer with experience in the following techologies:
Databricks and Azure
Apache Spark-based, hands on Python, SQL, Apache Airflow.
Databricks clusters for ETL processes.
Integration with ADLS, Blob Storage.
Efficiently ingest data from various sources, including on-premises databases, cloud storage, APIs, and streaming data.
Use Azure Key Vault for managing secrets.
Hands on experience working with API''s
Kafka/Azure EventHub streaming hands on experience
Hands on experience with data bricks delta API''s and UC catalog Hands on experience working with version control tools Github
Data Analytics
Supports various ML frameworks.
Integration with Databricks for model training.
OnPrem
Exposure on Linux based systems
Unix scripting