Design, implement, and own complex distributed features across the data and microservices ecosystem.
Build scalable and resilient Spark-based data processing pipelines (batch and/or streaming).
Develop and maintain microservices to support platform services and ML pipelines.
Work closely with architects, product managers, and collaborators to develop technical solutions from concept to production.
Optimize the performance, cost, and reliability of large-scale distributed systems.
Lead code reviews, mentor junior engineers, and enforce engineering standard methodologies.
Contribute to tooling and automation that support CI/CD, observability, and incident response.
Requirements
6+ years of professional software engineering experience.
Proven ability to lead features or systems end-to-end from design through implementation to deployment and monitoring.
Strong proficiency in Scala, Java, or Python (with Spark expertise in at least one).
Deep understanding of Apache Spark internals, tuning, and optimization (structured streaming is a plus).
Experience with microservices architecture and frameworks (e.g., Spring Boot, FastAPI, or similar).
Solid grasp of distributed systems concepts like consistency, availability, partitioning, and fault tolerance.
Hands-on experience with cloud platforms (AWS, GCP, or Azure) and orchestration tools (Docker, Kubernetes) or equivalent experience.
Familiarity with messaging/streaming platforms such as Kafka, Pulsar, or similar.
Nice to Have
Experience with data lake formats like Delta Lake, Iceberg, etc.
Understanding of data governance, lineage, and compliance requirements.
Exposure to machine learning infrastructure or MLOps pipelines.
Familiarity with Generative AI efficiency tools (e.g., GitHub Copilot, ChatGPT, code generation workflows).
Background or interest in machine learning models, data science workflows, or AI-powered systems.
Strong system design skills and interest in technical leadership.
Job Classification
Industry: IT Services & Consulting Functional Area / Department: Engineering - Software & QA Role Category: Software Development Role: Technical Architect Employement Type: Full time