I am working on a agentic solution for laon processing that is approve, reject or manual review. Criteria for the decision I am giving in the context document.
When I’m writing prompt to make decision to approve, reject, or manual review, it is working fine.
But when I’m adding additional steps to use tools such as escalation for manual review, submit for application etc. then agent is not returning correct decision.
It is not processing comparison operations correct such greater than, less than etc.
I have tried updating context documents, providing step by step explanation for calculation as well as examples, even tried rewiring the prompt but nothing seems working for me.
Try by separating both the work like create one agent for just decision making and 2nd agent for tools escalation to execute based on descision and execute them in Maestro
Agent is considering all the criterias given in context but only the comparison operation is incorrect, please see the attached screenshot of agent output.
When there is prompt only to determine the decision, it work perfect.
But as soon as I add steps to follow for each decision such as using tools like send email, submit loan application or escalation for manual review, it given incorrect decision.
as per screenshot of your prompt…when processing loan and if decision manual etc are criteria only…if there are contradicting criteria then it might give a wrong decision
I have gone through the documentation, created the prompt again with the help of autopilot but still the issue persists. If one criteria works other does not work correctly. Facing issues with only comparisons (<,>).
You might want to fine tune your prompt with a much more refined prompting technique like Chain of thought prompting technique, as that will help you define the goal into more logical steps that align with your objective. Please check out the System Prompt example and the guidelines around the Structure. This will give you an idea for a well defined system prompt.
Autopilot is a fantastic way to start, although use it as a baseline and customize it for your business goal.
Here’s another example of how clarity and the complete instructions contributes towards the prompt’s effectiveness:
I have updated the prompt and got the expected results.
But another challenge I am facing now:
Consumption limit of robot units is reached, hence I imported the solution to other tenant/organization. And again facing the same issue with same prompt, context and resources which were working perfect in previous tenant.
The issue is now resolved, I have built the agent from scratch again.
One thing I have noticed, if you generate the prompt using autopilot and make some changes in it, the agent will not give results as expected. You need to be very careful while changing autopilot generated prompt and try to make minimal changes in it.
