UiPath Agent Builder: Struggling with Non-Deterministic Behavior and Context File Ignoring

Help Agent Builder
1

Hi everyone,

I am currently building an agent using the new UiPath Agent Builder, but I am struggling heavily with its non-deterministic behavior. Despite giving very explicit instructions, the agent frequently diverges from my rules.

I am facing two specific issues and would love to know if anyone has found workarounds, or if this is a known limitation of the current preview/version.

:pushpin: Issue 1: Ignoring Phased Instructions in the System Prompt

In my System Prompt, I have explicitly defined a set of sequential Phases that the agent must follow to complete the task.

  • The Problem: In practice, the agent does not always follow these steps. It often skips phases or jumps straight to the end, making the execution unreliable.

:pushpin: Issue 2: Ignoring Specific Context Files

I have uploaded multiple files into the agent’s context. When I ask the agent to retrieve data or perform an action on one specific file (explicitly named), it ignores my instruction.

  • The Problem: The agent decides on its own to look at other files in the context that it “thinks” are more relevant, instead of obeying the direct file reference I provided.

:red_question_mark: My Questions to the Community:

  1. Has anyone else experienced this lack of determinism in Agent Builder?
  2. Is it currently possible to force a stricter, more deterministic execution flow in Agent Builder?
  3. Are there any prompt engineering tricks specific to UiPath’s implementation that help anchor the LLM to specific files or step-by-step phases?

Thanks in advance for any insights or help!

2

@yuvalB

One step that helped us and make it more deterministic is to use auto pilot to correct them explaining the issue

also in prompt if there are contradicting statements then mostly both are being ignored

Coming to context one thign we observed is if we use standard while indexing instead of advanced it works well with small file or with text files or json

with complex or large files standard is not giving good results

cheers

3

Hi @yuvalB

This is a very common trend that developers are seeing when building Agents. We call this “Hallucinating” - when the Agent diverges/ignores parts of the prompt.

These are some recommendations that brought my Agent accuracy up to 98%:

  1. With the context documentation that you provided, make reference to specific files where information is important or not, and put this in the System Prompt.
  2. Focus on BOTH “What to do” AND “What NOT to do” in the agent System Prompt - this helps the agent understand what it’s role is each runtime. Many people forget about the “What not to do” portion - and it causes the agent to hallucinate. Be strict, and be precise in your instructions, make sure the agent knows exactly what to say at all times.
  3. IMPORTANT - Use evaluation sets as often as possible. By building an evaluation set, and constantly training your agent on it, it will learn what responses were CORRECT and which were INCORRECT. This adds great value to your overall agent understanding.
  4. Make sure you are providing valid documentation in your context for indexing. Don’t add huge/long documents, don’t add duplicates, and don’t add older versions of same documentation where there might be conflicting definitions as an example.
  5. Lastly, I would recommend taking your existing prompt and pasting it in Autopilot, Claude or ChatGPT and requesting it to refine it in Agentic terms for your process. Prompt engineering is not as easy as we think - sometimes specific keywords or characters can make a big difference in your prompts.

If you use Agent Builder correctly, you can make a pretty snazzy Agent! :smiling_face_with_sunglasses:

Let me know if this helps :slight_smile:

4

hy,
can you please tell me is the agent learn form the evaluation set?
because, as I know, the evaluation set is done just to test the agent with an input and waited output, but the agent doesn’t learn from it, because the agent use a learned llm model.
I don’t get this point

5

Hi @Sal23

The Agent does not automatically learn or train in real time from evaluation sets. What I mean by this is that is can help you improve your Agent.

Basically evaluation sets are strictly used for testing and validating your agent behavior against expected outcomes. They act as a “golden set” of test cases, not as a machine learning training loop.

The output of the evaluation set will just help you understand if you’ve gone wrong with your initial prompts and training so that you can refine them as you go alone and then test again.