AI Agent Functional & Behavioral Testing
Budget: ₹37,500 – ₹75,000 INR
Functional & Behavioral Testing:
Test AI agents across multi-step workflows, goal completion, and decision-making paths
Validate agent behavior under normal, edge, and failure scenarios
Verify correct tool calling, API usage, and response handling
Ensure consistency across retries, sessions, and memory states
Reasoning & Safety Validation:
Evaluate agent reasoning quality (logic, coherence, hallucination risk)
Test compliance with safety, policy, and ethical(HIPAA) constraints.
Identify prompt injection, jailbreaks, or unintended autonomous actions
Automation & Evaluation:
Design and maintain automated test suites for agent workflows.
Create evaluation metrics for accuracy, latency, success rate, and failure recovery
Use logs, traces, and conversation graphs to debug agent behavior
Data & Feedback Loops;
Curate and maintain test datasets, test cases and scenarios
Label and analyze failures to improve prompts, policies, and agent design
Collaborate with ML, product, and engineering teams to close gaps.
Experience testing AI/ML or LLM-based systems
Understanding of prompting and prompt chaining,tool/function calling,state, memory, and context management
Good to Have skills:
Experience with LLM frameworks (LangChain, AutoGen, CrewAI, Semantic Kernel, etc.)
Familiarity with evaluation tools (LLM evals, red teaming, simulation testing)
Knowledge of observability tools (traces, logs, replay tools)
Exposure to security or safety testing
Please confirm if you are okay with above JD
Test AI agents across multi-step workflows, goal completion, and decision-making paths
Validate agent behavior under normal, edge, and failure scenarios
Verify correct tool calling, API usage, and response handling
Ensure consistency across retries, sessions, and memory states
Reasoning & Safety Validation:
Evaluate agent reasoning quality (logic, coherence, hallucination risk)
Test compliance with safety, policy, and ethical(HIPAA) constraints.
Identify prompt injection, jailbreaks, or unintended autonomous actions
Automation & Evaluation:
Design and maintain automated test suites for agent workflows.
Create evaluation metrics for accuracy, latency, success rate, and failure recovery
Use logs, traces, and conversation graphs to debug agent behavior
Data & Feedback Loops;
Curate and maintain test datasets, test cases and scenarios
Label and analyze failures to improve prompts, policies, and agent design
Collaborate with ML, product, and engineering teams to close gaps.
Experience testing AI/ML or LLM-based systems
Understanding of prompting and prompt chaining,tool/function calling,state, memory, and context management
Good to Have skills:
Experience with LLM frameworks (LangChain, AutoGen, CrewAI, Semantic Kernel, etc.)
Familiarity with evaluation tools (LLM evals, red teaming, simulation testing)
Knowledge of observability tools (traces, logs, replay tools)
Exposure to security or safety testing
Please confirm if you are okay with above JD