Data Engineer (Azure / AWS) @ ITSource

Job Description

Job Description: We are seeking a skilled Data Engineer with expertise in both Azure and AWS cloud platforms to join our dynamic team. This role involves designing, building, and optimizing our data architecture, as well as managing and maintaining our data pipeline infrastructure. Key Responsibilities: Data Architecture and Management: Design and implement scalable and secure data solutions on Azure and AWS. Ensure the architecture supports our business requirements and integrates seamlessly with our existing systems. Pipeline Development: Develop and maintain robust data pipelines for data ingestion, transformation, and distribution. Utilize Azure Data Factory, AWS Glue, and other relevant services. Data Storage and Processing: Implement and manage data storage solutions using Azure Blob Storage, AWS S3, and other data warehousing technologies like Azure SQL Data Warehouse and AWS Redshift. Performance Tuning: Optimize data retrieval and processing. Implement data. partitioning and indexing strategies to improve performance and scalability. Data Security: Ensure the integrity and security of data by implementing comprehensive data protection and encryption methods, adhering to compliance and privacy regulations. Monitoring and Maintenance: Monitor data pipelines and performance, troubleshoot issues, and perform regular maintenance to ensure optimal performance. Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand their data requirements and provide support. Continuous Improvement: Stay updated with the latest trends and technologies in cloud data services and implement best practices to enhance our data infrastructure. Qualifications: Bachelors degree in Computer Science, Engineering, or a related field. Proven experience as a Data Engineer, with hands-on experience in both Azure and AWS environments. Strong knowledge of SQL, Python, and other programming languages. Experience with big data tools like Hadoop, Spark, Kafka, etc. Familiarity with data modeling, ETL development, and data warehousing principles. Excellent problem-solving skills and ability to work in fast-paced environments. Strong communication and collaboration skills

Employement Category:

Employement Type: Full time
Industry: Others
Role Category: Engineering Design / Construction
Functional Area: Not Applicable
Role/Responsibilies: Data Engineer (Azure / AWS)

+ View Contact

Keyskills:   Azure AWS Data Architecture Data Management Performance Tuning Data Security Monitoring Maintenance Collaboration Continuous Improvement SQL Python Hadoop Spark Kafka Data Modeling Data Warehousing Communication Data Pipeline Azure Data Factory AWS Glue Azure Blob Storage AWS S3 Azure SQL Data Warehouse AWS Redshift ETL Development Problemsolving