As we know, we can add evaluation sets in the Agent Builder.
How do the evaluation sets help improve the agent’s performance?
I understand that the evaluation sets assist in handling edge cases and in deciding which model and temperature to use, but I am more interested in learning whether the agent actually trains in real time with the evaluation sets.
If a user adds 10 evaluation sets, would they notice an improvement before adding the evals?
Does the agent train on the evaluation sets, so is it advised to add as many evaluation sets as possible for agent training?
Evaluation sets are just used for testing. You cannot use them as a training mechanism. Evaluation sets allow you to run your agent against a collection of prompts provided (including edge cases) and compare the agent’s responses against expected outcomes.
Agents cannot be “trained” because they just use the available LLM’s like GPT, Claude, Gemini etc. Select the LLM that best fits your use case. However, to make your agent work better, you can just update the prompts.
But I don’t get why there is a positive and negative feedback trace system.
If it is only for testing, we can do that directly without eval sets.
I see that the agent health score uses eval sets to make results more reliable.
If my prompt is the same, and results don’t change with or without eval sets, why are eval sets part of the score?
If eval sets are only for testing, I don’t think they should affect agent performance.
It is said that we can manually set expected results in eval sets.
If this does not train the agent, then what is the point of doing it?
I also read about having 30+ eval sets, which makes things more confusing.
Considering all this overall, I’m confused a little about the exact link between eval sets and agent training/performance.
I think the score include the evaluation just to verify if there are any .
It just compare the number of set to the recommended length is >=30 and to
It doesn’t learn from the results
