Data Engineer - Bangalore
- HTC Global Services
- 5 to 10 Yrs
- Bengaluru
- 4 years ago
- Email to a friend
- Report this job
Job Description
- Own, design, build, operationalize, secure, and monitor data processing systems
- Translating business use cases across all products into technical specifications, including data streams, integrations, transformations, databases, and data warehouses
- Defining the data architecture framework, standards and principles, including modelling, metadata, security
- Build logical and physical data models in sustainable, scalable manner and Design various ETL processes defining data flows across data stores/systems
- Collaborate with Product Owner, IT, business process owners and data scientists to design/build/improve Data ware house and datamarts
- Be a Tech Anchor for the team on streaming & batch ETL development activities conducting code reviews, and resolving technical issues and dependencies
- Optimizing queries and building a scalable data model
- Monitor processes and develop plans to capture and access all metadata, according to various ETL processes.
- Evaluate all proposals requests and improve structure of data warehouse
- Ensure accuracy and quality of data movement across ETL pipelines in production environment
Skills Required:
6+ years of work experience in data warehousing data engineering or related field with experience in manipulating, processing, and extracting value from data sources
- Solid experience on building data lakes and data warehouses with various data sources on multiple projects
- Hands on experience on designing and building streaming ETL pipelines (using Kafka, Spark etc.,)
- Tools/Technology: Spark SQL/ Spark Streaming or Big Query (end-end); Apache Beam or Dataflow; Kafka or pub/sub
- Should have sound experience / knowledge on SQL Query optimization & performance tuning
- Hands on experience in programming languages like Java, Scala, Python (any two)
- Databases: SQL Server, PostgreSQL, MySQL (one or more required)
- Exposure to Hadoop ecosystem: HDFS, Hive, MapReduce, Oozy
- Must have knowledge of any public cloud platform like AWS, Azure, Google Cloud, etc.
Employement Category:
Employement Type: Full timeIndustry: IT - Software Role Category: Application Programming / Maintenance Functional Area: Not ApplicableRole/Responsibilies: Data Engineer - BangaloreContact Details:
Company: HTC Global ServicesLocation(s): Bengaluru
Keyskills: data engineering java spark python kafka
Similar positions
HTC Global Services
HTC Global Services (India) Pvt. Ltd.Established in 1990, HTC is a leading global IT Solutions and Business Process Outsourcing (BPO) provider with headquarters in Troy, Michigan. HTC is a mid-sized IT company with over 6000 qualified and experienced professionals. HTC has a strong client base ...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in