Data Engineer
- Omnicom Media Group
- 0 - 5 years
- Hyderabad
- 8 months ago
- Email to a friend
- Report this job
Job Description
Job Title: Data Engineer
Location:
Position: Full-time
Department: Annalect Engineering
Position Overview
Annalect is currently seeking a data engineer to join our technology team. In this role you will build Annalect products which sit atop cloud-based data infrastructure. We are looking for people who have a shared passion for technology, design & development, data, and fusing these disciplines together to build cool things. In this role, you will work on one or more software and data products in the Annalect Engineering Team. You will participate in technical architecture, design, and development of software products as well as research and evaluation of new technical solutions.
Key Responsibilities:
- Design, build, test and deploy scalable and reusable systems that handle large amounts of data.
- Collaborate with product owners and data scientists to build new data products.
- Ensure data quality and reliability.
Required Skills
- Experience designing and managing data flows.
- Experience designing systems and APIs to integrate data into applications.
- 4+ years of Linux, Bash, Python, and SQL experience
- 2+ years using Spark and other frameworks to process large volumes of data.
- 2+ years using Parquet, ORC, or other columnar file formats.
- 2+ years using AWS/GCP/Azure cloud services, esp. services that are used for data processing e.g. Glue, Dataflow, Data Factory, EMR, Dataproc, HDInsights , Athena, Redshift, BigQuery etc.
- Passion for Technology: Excitement for new technology, bleeding edge applications, and a positive attitude towards solving real world challenges.
Additional Skills
- BS, MS or PhD in Computer Science, Engineering, or equivalent real-world experience (You've learned something to be able to claim you are an engineer)
- Significant experience with Python, C++, or other popular language
- Experience with big data and/or infrastructure. Bonus for having experience in setting up Petabytes of data so they can be easily accessed. Understanding of data organization, i.e., partitioning, clustering, file sizes, file formats. Data cataloging with Hive/Hive metastore or Glue or something similar.
- Experience working with relational databases.
- Experience with Hadoop, Hive, Spark, or other data processing tools
- Experience building scalable data pipelines (Airflow experience a plus)
- Significant experience working with AWS and/or GCP.
- Proven ability to independently execute projects from concept to implementation to launch and to maintain a live product.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Machine LearningRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Omnicom Media GroupLocation(s): Hyderabad
Keyskills: Pyspark Data Engineering AWS SQL Python
Similar positions
Omnicom Media Group
At Annalect, we work with clients to develop data driven marketing strategies, powered by a connected system of technology, tools, consultants and digital activation. Drawing from the vast resources of our parent company Omnicom and spanning the globe in a variety of industry verticals, we are the d...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in