Data Engineer
- Vigoursoft Global
- 4 - 9 years
- Bengaluru
- 3 months ago
- Email to a friend
- Report this job
Job Description
- Expert in building and scaling data pipelines for ML/GenAI using Spark, Airflow, PostgreSQL, and cloud-native practices.
- Must have experience to handle end-to-end GenAI projects: NLP model usage, embeddings, pipelines
- Hands on experience with LLMs with Spark/Airflow ETL
- Hands-on experience with tool stack: GitHub, Docker, FastAPI, Redis, Ray Serve, Kubernetes and GitHub Actions.
- Should have experience in tokenization and embedding generation work using Hugging Face.
- Should be able to generate prompts for the GPT and Should be able to fine-tune GPT for a classification task.
- Should have experience in building a RAG pipeline using LangChain + FAISS
- Should be able to integrate FastAPI with OpenAI function-calling
- Should have knowledge of Slackbot agent and its modules, Tool-Calling Agent, Multi-Agent System, License Optimizer Bot etc.
- Should be able to design a resilient Spark pipeline to process SFDC PostgreSQL data for NLP.
- Candidate must have a deep understanding of Postgres and DB design practices.
- Candidate must have experience with CI/CD pipeline to deploy and monitor an LLM service.
- Candidate should take initiative and ownership of cross-functional projects and run projects with limited supervision
- Candidate should be innovative, able to scale ideas, and should have vision in the latest in GenAI/ML/Data Engineering.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Vigoursoft GlobalLocation(s): Bengaluru+ View Contact
Keyskills: Airflow Generative Ai Spark Python SQL
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Vigoursoft Global
Founded in 2016, VigourSoft Global Solutions is a Big data analytics based software product engineering services firm specializing in Customer Data Platforms, MarTech, AdTech, Infrastructure, Healthcare and Business Analytics. We help develop cloud based products and platforms that improve market sh...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in