Job Description
Responsibilities: Data Architecture Design: Develop and refine data architecture blueprints to support scalability and performance while aligning with the organization's strategic objectives. Pipeline Development: Lead the design and implementation of data pipelines, ensuring data ingestion, transformation, and storage are efficient, reliable, and compliant with data governance standards. Performance Optimization: Continuously optimize data processes, databases, and ETL workflows to enhance system performance, reduce latency, and minimize downtime. Data Governance: Enforce data governance policies and standards, including data quality, security, and compliance, while collaborating with data stewards and data governance teams. Team Leadership: Mentor and guide junior data engineers, providing technical expertise, and fostering a culture of best practices and innovation within the team. Big Data Technologies: Stay up-to-date with emerging big data technologies, evaluate. their applicability, and lead the adoption of relevant tools and frameworks. Data Modeling: Design and maintain data models that facilitate efficient data retrieval and support the needs of data analysts and data scientists. Analyze, clean and organize raw data from disparate sources, like CSV files, external API endpoints and internal databases. Data Integration: Collaborate with cross-functional teams to integrate data from various sources, including APIs, databases, and external services, into the data ecosystem. Monitoring and Troubleshooting: Implement robust monitoring solutions to proactively identify and resolve data-related issues, ensuring data availability and reliability. Requirements: Great numerical and analytical skills. 6+ years experience as a data engineer (or in a similar role). Deep knowledge of SQL. Excellent communication skills, to work closely with peers and leaders throughout the organization. Advanced technical knowledge in Software Engineering using at least one of the following programming languages: Python, Java, Node. JS. Experience with data pipeline orchestrators like Airflow, Dagster, Prefect, etc. Hands-on experience managing large PostgreSQL databases. Previous experience building and maintaining Data Lakes. Familiarity with AWS cloud services
Employement Category:
Employement Type: Full time
Industry: Others
Role Category: General / Other Software
Functional Area: Not Applicable
Role/Responsibilies: Senior Data Engineer
Keyskills:
Pipeline Development
Performance Optimization
Data Governance
Team Leadership
Data Modeling
Data Integration
SQL
Python
Java
Airflow
PostgreSQL
Data Architecture Design
Big Data Technologies
Monitoring
Troubleshooting
Nodejs
Dagster
Prefect
AWS Cloud Services