AgentHack submission type
Enterprise Agents
Name
Manav Mogha
Team name
TriggersMind
Team members
@Manav @Gargi @Tushar @Tejashri
How many agents do you use
One agent
Industry category in which use case would best fit in (Select up to 2 industries)
Banking and financial services
Information technology and services
Complexity level
Advanced
Summary (abstract)
We’ve put in place a strong agentic automation solution utilizing UiPath’s Document Understanding, Agent Builder, Apps, and Action Center to address the difficulties of downloading various invoice types from a website that changes frequently. This method makes it possible for:
Data extraction using generative document understanding from a variety of invoice formats.
Self-healing automation that automatically updates the selector in response to UI changes, ensuring that the bot reruns automatically the next time with no selector-related exceptions.
Validation by a human in the loop for data with low confidence or broken selectors
Real-time configuration updates and recovery via user input
Enhanced data quality, resilience, and scalability
By switching from static scripts to adaptive automation, the system is future-proofed for changing business requirements, improves process uptime, and reduces manual efforts by over 90%.
Detailed problem statement
At present, we have to download invoices in many formats from a website with an often changing UI. This is a manual download that is quite time consuming too, as there are multiple invoice types, each with their own format/layout and data structure. Team members typically spend a good portion of their time navigating the website to get to the appropriate invoice type and downloading them one-by-one.
Current Automations:
We have limited, if at all, automations in place - mainly, basic automations, like scripts that usually look at a fixed UI element or static page structure. Then, if the website structure changes, then it breaks the automations again requiring a manual intervention and automation updates, although they are straightforward and accounting operations are simple to automate.
Challenges in Achieving Agentic Automation:
UI Instability: The website has frequent UI changes and, therefore the platform has little automation stability from session-to-session. Automation using traditional scripts or UI-based bots, requires lots of continuous work and rework for new and intermittent changes.
Multiple Invoice Types: Each format of invoice may require a different logic in terms of extraction, which complicates any generalized approach to automation.
Human Dependency: The process remains very manual, lacking adaptive automation and intelligent agents that can adapt to UI changes in a timely manner, or recognize the potential for additional invoice formats.
Scalability Risks: The ability for maintainability decreases as the volume and variety of invoices increase; within the existing automation context, sustainment is neither efficient or effective, in maintenance nor processing times.
A transition towards agentic automation will require a much more resilient and intelligent mechanism for working to comprehend UI objects, flexible processes and multiple invoice templates with minimal reconfiguration.
| Feature / Capability | Healing Agent | Our Selector Fixing Solution |
|---|---|---|
| Technology | Built-in UiPath Auto Healing | UiPath Agent Builder, Apps, and Action Center |
| Triggering Mechanism | Auto fix at runtime | Proactively & reactively detects failed selectors |
| Scope of Fix | Finds similar UI element | Refreshes page, inspects DOM, validates selector |
| Fallback Mechanism | Logs error if fix fails | Triggers UiPath Apps form for manual correction |
| Human-in-the-loop | No | Yes – user provides corrected selector in Apps |
| Config Update After Fix | No – fix not saved | Yes – updates config file with corrected selector |
| Openness of Error | Low – silent or logged error | High – shows error, selector path & exception |
| Learning Ability | No memory of past fixes | Yes – learns from and stores validated selectors |
| Customization / Control | Limited – uses UiPath heuristics | Full control over fix logic, retries, escalation |
| Scalability for Complex UI Flows | Limited | Good – handles dynamic UIs with fallback paths |
Detailed solution
As a remedy for the difficulties of downloading and processing several different types of invoices while facing a constantly changing website UI, we put together a robust agentic automation solution using the capabilities of UiPath - and we implemented four elements of UiPath relative to Document Understanding, Agent Builder, UiPath Apps, and Action Center.
- Addressing Multiple Invoice Types with Document Understanding
We are utilizing UiPath Document Understanding to process and retrieve structured data from all different invoice formats. Document Understanding enables the automation to intelligently classify and extract the data regardless of how the layout may vary, thus limiting the requirement of building scripts for each layout.
Flexibility: The trained machine-learning models will resolve the variations that may arise in invoice structure.
Accuracy: Confidence standard ensures proven data reliability, and low confidence data points will be flagged for validation.
- Addressing UI Changes Using UiPath Agent Builder
As the website UI changes very frequently, we have implemented Agent Builder to build a oblivious (robust) and adaptive automation layer:
Dynamic Selector Correction: The agent will detect invalid or stale selectors, and will on its best effort; tries to correct the selector by refreshing the page and extracting the DOM to see if the selector is again valid.
Validation Mechanism: If the agent is unable to correct the selectors - it will raise a UiPath Apps Action in Action Center for human action.
- Human-in-the-Loop Validation through UiPath Apps & Action Center
When automated selector correction does not work:
A UiPath Apps form opens up that will display:
The incorrect selector
The system exception
A manual input block for the user to enter the corrected selector
A robot will then update the configuration file with the new selector ensuring that subsequent runs will not encounter the same issue.
This human-in-the-loop method aligns well with organisational practices in allowing for the least disruption and fast recovery when UI changes occur.
- Confidence-Based Data Validation & Storage Bucket
Similarly, when extracted data from invoices does not surpass a defined confidence threshold, a Validation Action is triggered in Action Center.
The corresponding invoice is stored in a Storage Bucket so that either users or clients can manually review and validate the data.
As such, you guarantee data quality without stopping the automation pipeline.
Key Benefits of This Solution:
Agentic Automation: Automate ethical behaviour in association with agents capable of self-healing and recursive behaviour in response to UI changes, thus mitigating manual script maintenance.
Scalability: Handle multiple invoice types while achieving minimal configuration OS overhead.
Resilience: Automatically manage unpredictable UI behaviour with fallback processes.
Demo Video
Expected impact of this automation
Implementing our agentic automation solution—s Utilizing UiPath’s Agent Builder, Document Understanding, Apps, and Action Center—will result in a fundamental change in the current system through the following three primary components:
-
Materially Reduce Manual Workload
The agentic automation dramatically reduces the amount of manual downloading, sorting, and validating of different invoice documents
Human participants are committed only to exception scenarios or low-confidence cases, reducing manual effort overall by 60% to 80%. -
Increased Resilience to UI Changes
With the Agent Builder allowing for dynamic adjustments and corrections to selectors, the agentic automation now has increased resilience to frequent user interface (UI) changes.
This reduces interruptions of process and performs; and decreases maintenance responsibilities inherent with UI bots . -
Improved Data Quality and Compliance
Using confidence scoring, and validation actions, we are assured that quality, verified data is only processed.
The storage buckets and human review provide an audit trail for compliance, and attach a level of data traceability. -
Improved Resolution and Self-Healing
Selector-related problems will now be resolved in real time by agents or involve a quick (or quick) handoff for human correction.
This reduces recovery time in the case of errors from hours, or days or longer, down to minutes providing an increase in overall uptime for the process. -
Scalability and Flexibility
The system will process new invoice formats in an easy-to-configure way making it scalable for future growth or changes in business needs. -
Improved user and business confidence.
The transparency of fallback mechanisms (like Action Center, and Apps-based correction) provides more user trust in the automation system.
It also advances human + digital worker collaboration which is an important principle of agentic automation.
UiPath products used (select up to 4 items)
UiPath Action Center
UiPath Agent Builder
UiPath Apps
UiPath Maestro
Automation Applications
Edge Browser, Excel
Integration with external technologies
N/A
Agentic solution architecture (file size up to 4 MB)
Sample inputs and outputs for solution execution
Inputs: Config File
Output: Correct the incorrect selector using agent and give the extracted data from the invoice using LLM.
Other resources
Video Link - “https://drive.google.com/file/d/1sE62HO9PhzV9HeZxYXfazlXibV23TzC5/view?usp=drive_link”
PPT link - “https://1drv.ms/p/c/641c0c224791f24d/EX9QEZ30X0VKgDxseez00_0B6dyPaRKiC2VSG8Sd3jplWQ?e=l19xZc”