Senior Data Engineer
- Ifood Web Media
- 5 - 8 years
- Chennai
- 5 months ago
- Email to a friend
- Report this job
Job Description
Job Title: Senior Data Engineer OTT Streaming Platform
Location: [ Remote]
Key Responsibilities
1. Data Pipeline Development & Maintenance
- Design, implement, and maintain scalable ETL/ELT pipelines for ingesting, processing, and transforming structured, semi-structured, and unstructured data from multiple sources (playback events, ad servers, content catalogs, third-party APIs).
- Ensure real-time streaming data ingestion for analytics, personalization, and A/B testing using tools like Kafka, Kinesis, or Pub/Sub.
- Build batch and hourly refresh pipelines for operational reporting, audience insights, and decision support.
- Implement AI-driven data processing workflows to optimize transformation logic, resource usage, and anomaly detection.
2. Data Modeling & Warehousing
- Develop and maintain optimized data models for content performance, audience segmentation, churn prediction, and monetization analysis.
- Manage cloud-based data warehouses (Snowflake, BigQuery, Redshift, or Databricks) for high-volume analytics workloads.
- Utilize AI-based optimization for schema design, query acceleration, and automated indexing to enhance performance and reduce costs.
- Ensure schema design aligns with analytical and operational needs.
3. Data Quality, Governance & Security
- Implement data validation, monitoring, and alerting frameworks to ensure accuracy, consistency, and availability of datasets.
- Integrate AI-enabled data quality monitoring for proactive detection of inconsistencies, delays, or corruptions.
- Work with the Data Governance team to maintain compliance with GDPR, CCPA, and other privacy regulations.
- Enforce secure access controls for sensitive user and business data.
4. Collaboration & Business Impact
- Partner with Data Science and ML teams to prepare training datasets for recommendation algorithms, ad targeting, and personalization models.
- Collaborate with Product and Marketing teams to deliver audience insights and campaign performance metrics.
- Drive AI adoption across data engineering workflows to improve agility, scalability, and decision-making speed.
- Provide technical mentorship to junior engineers.
Qualifications Required:
- Bachelors/Masters degree in Computer Science, Data Engineering, or related field.
- 6+ years of experience in data engineering roles, preferably in high-scale consumer tech, media, or OTT streaming environments.
- Strong programming skills in Python, Scala, or Java for data processing.
- Hands-on experience with distributed data frameworks (Apache Spark, Flink, Beam).
- Proven expertise with streaming data technologies (Kafka, Kinesis, Pub/Sub).
- Proficiency in SQL and performance optimization for large datasets.
- Experience with cloud platforms (AWS, GCP, Azure) and services like S3, EMR, Dataflow, Glue, etc.
- Strong understanding of data modeling (star/snowflake schemas) and warehousing best practices.
- Practical experience in applying AI/ML techniques for data pipeline optimization, automated schema evolution, query performance tuning, and anomaly detection.
Preferred:
- Experience with OTT/Media streaming analytics (e.g., playback logs, CDN metrics, ad impression data).
- Familiarity with recommendation systems, A/B testing frameworks, and audience segmentation.
- Knowledge of ad tech pipelines for AVOD/FAST (VAST, SSAI, client-side tracking).
- Exposure to machine learning data pipelines and MLOps practices.
Key Skills & Attributes
- Ability to work with petabyte-scale datasets with performance and cost-efficiency in mind.
- Strong analytical thinking with a business impact mindset.
- Excellent communication skills for cross-functional collaboration.
- Self-driven, proactive, and able to work in fast-paced environments.
- Demonstrated ability to blend traditional data engineering with AI-powered automation.
Performance Metrics
- Data pipeline uptime and latency.
- Data accuracy and completeness.
- Query performance improvements.
- AI-enabled efficiency gains in data processing and warehouse management.
- Contribution to business KPIs (e.g., CLV, data warehouse, feature impact, A&B testing etc ).
Job Classification
Industry: Internet (OTT)Functional Area / Department: Data Science & AnalyticsRole Category: Data Science & Machine LearningRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Ifood Web MediaLocation(s): Chennai+ View Contact
Keyskills: Bash PHP Python Java Scala
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Ifood Web Media
Important Links that you can look up are: http://futuretodayinc.com/ (parent company) https://fawesome.tv/ https://happykids.tv/ About the Company: We are an OTT video streaming company, which specializes in cutting edge media technology and marketing. iFood Web Media Technology is the Indian ar...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in