LLM Ops Engineer
- Xoriant
- 7 - 12 years
- Mumbai
- 1 month ago
- Email to a friend
- Report this job
Job Description
Senior Software Engineer/ LLM Ops Engineer
External DescriptionDescription - External
JD -
What You Will Do
- Design, implement, and maintain LLM operations workflows using tools like Langfuse to monitor performance, track usage, and create feedback loops for continuous improvement
- Develop and maintain infrastructure-as-code for AI deployments using Terraform and AWS services (Lambda, SQS, API Gateway, OpenSearch, CloudWatch)
- Build and enhance monitoring, logging, and alerting systems to ensure optimal performance and reliability of our LLM infrastructure
- Collaborate with AI engineers to design and implement evaluation frameworks (including LLM-as-judge systems) to measure and improve model performance
- Manage prompt versioning, testing, and deployment pipelines through CI/CD and custom tooling
- Implement and maintain security guardrails for LLM interactions, ensuring compliance with best practices
- Create comprehensive documentation for LLM operations, including runbooks for production incidents
- Participate in on-call rotations to support mission-critical AI systems
- Drive innovation in LLM operations by researching and implementing best practices and emerging tools in the rapidly evolving GenAI space
- Deep understanding of prompt engineering strategies
What You Will Bring
To succeed in this role, you will need a combination of experience, technology skills, personal qualities, and education.
Required Qualifications
- 3+ years of experience in DevOps, SRE, or similar roles, with at least 1 year specifically working with LLMs or AI systems in production
- Strong hands-on experience with AWS cloud services, particularly Bedrock, Lambda, SQS, API Gateway, OpenSearch, and CloudWatch
- Experience with infrastructure-as-code using Terraform, CloudFormation, or similar tools
- Proficiency in Python and experience building automation tooling and pipelines
- Familiarity with LangOps platforms such as Langfuse for LLM observability and evaluation
- Experience with CI/CD pipelines
- Knowledge of logging, monitoring, and alerting systems
- Understanding of security best practices for AI systems, including prompt injection mitigation techniques
- Excellent troubleshooting and problem-solving skills
- Strong communication skills and ability to work effectively with cross-functional teams
- Must be legally entitled to work in the country where the role is located
Preferred Qualifications
- Experience with prompt engineering and testing tools like Promptfoo
- Familiarity with vector databases and retrieval-augmented generation (RAG) systems
- Knowledge of serverless architectures and event-driven systems
- Experience with AWS Guardrails for LLM security
- Background in data engineering or machine learning operations
- Understanding of financial systems and data security requirements in the finance industry
- Familiarity with implementing technical solutions to meet compliance requirements outlined in SOC2, ISAE 3402, and ISO 27001
Job Classification
Industry: Software ProductFunctional Area / Department: Engineering - Software & QA
Role Category: DevOps
Role: DevOps Engineer
Employement Type: Full time
Contact Details:
Company: XoriantLocation(s): Mumbai
Keyskills: prompt engineering strategies Langfuse retrieval-augmented generation LLM Devops
Similar positions
Xoriant
Xoriant Corporation is a product engineering and services company, serving technology startups as well as mid-size to large corporations. We offer a flexible blend of onsite, offsite and offshore services from our Global delivery centers ( Sunnyvale, New Jersey, Mumbai, Pune, Gurgaon, Kolkata and B...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in