AI Testing Consultant
- Trigyn Technologies
- 10 - 16 years
- Mumbai
- 17 hours ago
- Email to a friend
- Report this job
Job Description
Job Description AI Testing Consultant
Experience Required: 10 Years
Location: RemoteRole SummaryWe are looking for an AI Testing Consultant with handson experience in AI Agentbased solutions. The resource will be responsible for validating AI agents, testing workflows, and using Microsoft AI Foundry evaluators to measure quality, accuracy, and performance across different AI scenarios.Key Responsibilities
- Perform testing and validation of AI Agents, including singleagent and multiagent workflows.
- Execute functional, nonfunctional, and scenariobased testing for AI-driven use cases.
- Use Microsoft AI Foundry evaluators to assess AI performance metrics such as accuracy, relevance, grounding, completeness, and safety.
- Analyze evaluator results and help identify gaps, risks, and improvement areas in AI responses.
- Validate AI outputs against business requirements and expected behaviors.
- Work closely with onshore teams, architects, and developers to support test strategy and defect resolution.
- Document test scenarios, test results, and evaluation findings clearly.
Required Skills & Experience
- 10 years of experience in testing or validation roles, preferably in AI, ML, or intelligent automation.
- Handson experience testing AI agents, chatbots, or GenAI-based applications.
- Good understanding of AI evaluation metrics (e.g., precision, recall, relevance, groundedness, hallucination).
- Exposure to Microsoft AI Foundry evaluators or similar AI evaluation frameworks.
- Understanding of prompt-based systems and agent behaviors.
- Strong analytical skills to interpret AI evaluation results.
- Good communication and documentation skills.
Nice to Have
- Experience with Copilot Studio or agentic AI architectures.
- Familiarity with testing multi-agent or orchestrated AI workflows.
- Experience working in enterprise environments with governance and compliance requirements.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Quality Assurance and TestingRole: Quality Assurance and Testing - OtherEmployement Type: Full timeContact Details:
Company: Trigyn TechnologiesLocation(s): Mumbai
Keyskills: AI Testing Azure AI Foundary Agentic AI Quality Assurance AI agent Testing
Similar positions
Trigyn Technologies
Trigyn Technologies ltd Trigyn is an ISO 9001:2008 certified, and CMMI Level 5 assessed company. Trigyn Technologies is an IT Services Company that has been in business for more than 30 years. Trigyn provides IT Solutions, Consulting, Systems Integration, Managed Services, Software Development, ...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in