I have created an Evaluator Set for my UiPath Agent containing the following evaluators:
Exact Match Evaluator
JSON Evaluator
Trajectory Evaluator
LLM as a Judge Evaluators
As shown in my Evaluator Set dashboard, the evaluators have execution results and scores.
However, when I navigate to the Deployed Agent → Evaluations (Preview) → Evaluators, I see the message “No evaluators”. Even after running the deployed agent multiple times, my evaluators are not appearing in this section.
I would like to understand:
Why are the evaluators configured in my Evaluator Set not appearing under the deployed agent’s Evaluations section?
Is there an additional step required to link or publish an Evaluator Set to a deployed agent?
Do evaluators only appear when evaluations are triggered through a Dataset or Evaluation Run, rather than normal agent executions?
How can I view the evaluation results for my deployed agent executions?
What is the difference between project-level evaluation and deployment-level evaluation?
How can project-level evaluations be practically used in a UiPath Maestro use case? For example, if I have a multi-agent Maestro process, can evaluators validate the output or trajectory of individual agents such as:
Triage Agent
Classification Agent
Warranty Validation Agent
Vendor Service Agent
I would appreciate clarification on the correct workflow for configuring evaluators, associating them with a deployed agent, running evaluations, and using evaluation results in an Agentic Automation and UiPath Maestro implementation.
The Evaluator Set results are separate from normal Agent executions.
Running the deployed Agent normally will not make the evaluators appear under Evaluations → Evaluators.
You need to create an Evaluation Set, add your test cases and evaluators, and run the evaluation. The results will then be available under the evaluation results.
For Maestro, you can create separate evaluation sets to test individual agents and validate their output or trajectory.
Hi @vikasbaburao.sajjan ,
The evaluators in your Evaluator Set are working at the project level, which is why you can see scores in the Evaluator Set dashboard. However, they do not automatically appear under a deployed agent’s Evaluations (Preview) tab.
The important points are:
Project-level evaluation = Used to test and benchmark an agent against datasets, expected outputs, trajectories, and prompts during development.
Deployment-level evaluation = Used to evaluate executions of a deployed agent. Evaluators must be explicitly associated with the deployed agent .
Simply running the deployed agent does not create evaluation runs automatically.
The “No evaluators” message usually means no deployment-level evaluators or evaluation runs are configured for that deployed agent.
Correct workflow you can create:
Create an Evaluator Set.
Run evaluations against a Dataset/Evaluation Run.
Review scores and optimize prompts/agent behavior.
Deploy the agent.
Configure deployment-level evaluators (if available in your tenant/version) and run evaluation jobs against the deployment.
Maestro use case
Project-level evaluators are very useful in Maestro for validating individual agent outputs:
Triage Agent → check routing accuracy.
Classification Agent → Exact Match or LLM Judge validation.
Vendor Service Agent → response quality and trajectory evaluation.
This lets you test each agent independently before combining them into an end-to-end Maestro workflow.
Why you see results in Evaluator Set but not in Deployed Agent
The screenshots indicate that the evaluators were executed as part of an Evaluator Set run, not as deployment evaluations. Therefore, the results appear in the Evaluator dashboard but not in Deployed Agent → Evaluations (Preview).
