Data Architect - Scala
- Happiest Minds
- 10 - 15 years
- Bengaluru
- 4 days ago
- Email to a friend
- Report this job
Job Description
Job Summary
Senior Data Architect with 10+ years of experience to design, build, and optimize scalable data pipelines and streaming solutions. The ideal candidate should have strong expertise in Spark (Streaming & Scala), Databricks, Delta Lake, Airflow, Snowflake, and AWS services, with a solid understanding of data engineering best practices and distributed systems.
Key Responsibilities
- Design and develop scalable batch and real-time data pipelines using Spark (Scala) and Spark Streaming
- Build and manage data workflows using Airflow
- Develop and optimize data solutions on Databricks with Delta Lake
- Integrate and manage data across Snowflake and AWS ecosystem
- Work with AWS services such as S3, ECS, and MSK (Kafka) for data ingestion and processing
- Ensure data quality, reliability, and performance of pipelines
- Collaborate with cross-functional teams to understand data requirements and deliver solutions
- Implement best practices for data governance, security, and privacy (CCPA/GDPR)
- Troubleshoot and optimize performance issues in large-scale distributed systems
- Mentor junior engineers and contribute to architectural decisions
Required Skills
- 10+ years of experience in Data Engineering / Big Data
- Strong hands-on experience with:
- Apache Spark (Scala) & Spark Streaming
- Databricks & Delta Lake
- Apache Airflow
- Snowflake
-
- Solid experience with AWS services:
- S3, ECS, MSK (Kafka)
-
- Strong programming skills in Scala/Java or Python
- Deep understanding of distributed data processing and ETL/ELT design patterns
- Experience in building high-performance, scalable data pipelines
Good to Have
- Experience with Datadog for monitoring and observability
- Working knowledge of Java and/or Python
- Familiarity with SBT (Scala Build Tool)
- Experience with GitHub Actions for CI/CD pipelines
- Understanding of data privacy regulations (CCPA, GDPR)
- Experience in real-time streaming architectures
Soft Skills
- Strong problem-solving and analytical skills
- Excellent communication and stakeholder management
- Ability to work in a fast-paced, collaborative environment
Preferred Qualifications
- Experience in cloud-native data platforms
- Prior experience in handling large-scale data platforms
- Exposure to data governance and compliance frameworks
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: DBA / Data warehousingRole: Data warehouse Architect / ConsultantEmployement Type: Full timeContact Details:
Company: Happiest MindsLocation(s): Bengaluru+ View Contact
Keyskills: Architecture SCALA data governance Data processing Data quality Stakeholder management Apache Distribution system Monitoring Python
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in