Databricks Administrator - AWS /Azure
- Numeric Technologies
- 8 - 13 years
- Hyderabad
- 10 days ago
- Email to a friend
- Report this job
Job Description
Databricks Administrator Azure/AWS | Remote | 6+ Years
Job Description:
We are seeking an experienced Databricks Administrator with 6+ years of expertise in managing and optimizing Databricks environments. The ideal candidate should have hands-on experience with Azure/AWS Databricks, cluster management, security configurations, and performance optimization. This role requires close collaboration with data engineering and analytics teams to ensure smooth operations and scalability.
Key Responsibilities:
- Deploy, configure, and manage Databricks workspaces, clusters, and jobs.
- Monitor and optimize Databricks performance, auto-scaling, and cost management.
- Implement security best practices, including role-based access control (RBAC) and encryption.
- Manage Databricks integration with cloud storage (Azure Data Lake, S3, etc.) and other data services.
- Automate infrastructure provisioning and management using Terraform, ARM templates, or CloudFormation.
- Troubleshoot Databricks runtime issues, job failures, and performance bottlenecks.
- Support CI/CD pipelines for Databricks workloads and notebooks.
- Collaborate with data engineering teams to enhance ETL pipelines and data processing workflows.
- Ensure compliance with data governance policies and regulatory requirements.
- Maintain and upgrade Databricks versions and libraries as needed.
Required Skills & Qualifications:
- 6+ years of experience as a Databricks Administrator or in a similar role.
- Strong knowledge of Azure/AWS Databricks and cloud computing platforms.
- Hands-on experience with Databricks clusters, notebooks, libraries, and job scheduling.
- Expertise in Spark optimization, data caching, and performance tuning.
- Proficiency in Python, Scala, or SQL for data processing.
- Experience with Terraform, ARM templates, or CloudFormation for infrastructure automation.
- Familiarity with Git, DevOps, and CI/CD pipelines.
- Strong problem-solving skills and ability to troubleshoot Databricks-related issues.
- Excellent communication and stakeholder management skills.
Preferred Qualifications:
- Databricks certifications (e.g., Databricks Certified Associate/Professional).
- Experience in Delta Lake, Unity Catalog, and MLflow.
- Knowledge of Kubernetes, Docker, and containerized workloads.
- Experience with big data ecosystems (Hadoop, Apache Airflow, Kafka, etc.).
Email : Hr**************a@nu********h.com
Phone /Whatsapp : 9700111***
For immediate response and further opportunities, connect with me on LinkedIn: https://www.linkedin.com/in/hrushikesh-a-74a32126a/
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Data Engineer
Employement Type: Full time
Contact Details:
Company: Numeric TechnologiesLocation(s): Hyderabad
Keyskills: Databricks S3 Ci Cd Pipeline Aws Cloud Notebook scala libraries Azure Databricks Arm Templates SQL Terraform Job Scheduling ARM AWS Python
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in