Data Engineers For Big Data Program
- Grid Dynamics
- 3 - 6 years
- Remote
- 12 months ago
- Email to a friend
- Report this job
Job Description
- A passion for continuous learning and professional growth
- Willingness to invest time in education and stay updated with the latest technologies
- Ability to work in a collaborative and dynamic environment
If you are ready to advance your career in Big Data , apply now and be part of an exciting journey towards becoming a Big Data expert!
Essential functions
- Uploader (Go written binary client with its backend API to transfer data from third-party into platform)
- Scheme registry (It is a metadata store with dataset schemas including schema management capabilities)
- Pipeline manager (Manage ETL pipelines: CRUD and execute. Triggers ETL pipeline on upload event Pipelines metadata storage)
- (Airflow DAG) data ingestion pipeline including data decryption, validation (optionally) and ingestion.
- Monitor data metrics. uploads and integrity. At least data size, num objects, uploader entity, path.
- Hone the soft skills necessary for effective collaboration with customer team members and stakeholders, including engineering managers.
Qualifications
- Strong Scala, Java or Python (Primary skill requirement)
- Strong SQL
- Good knowledge of Apache Spark
Would be a plus
- Python and/or Java is nice-to-have
- Snowflake, docker, K8s, Helm, PostgreSql, Superset/Tableau
- AWS(EKS, IAM, S3, SNS, SQS, MSK etc)
- Theoretical knowledge of Big Data concepts
- Commercial experience, working with real clients (not only pet-projects or RnD or Internships)
- Eager to invest his/her time to education and learn new technologies
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Big Data EngineerEmployement Type: Full timeContact Details:
Company: Grid DynamicsLocation(s): Remote+ View Contact
Keyskills: metadata Backend Development Manager spark Postgresql Schema SCALA big data SQL Python
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Grid Dynamics
NextSphere is full-service custom application development firm that helps customers grow and keep up, in a constantly changing technology landscape. We at NextSphere develop and support business applications for customers in wide range of industries. We strive to work on projects where the NextSpher...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in