AgentHack submission type
Agentic Testing Solutions
Name
Dhruba Jyoti Kalita
Team name
SoloPath
How many agents do you use
Multiple agents
Industry category in which use case would best fit in (Select up to 2 industries)
Information Technology(IT) services
Customer Service
Complexity level
Intermediate
Summary (abstract)
This solution is an intelligent test management system built using UiPath Agentic Automation. It autonomously detects anomalies, flags flaky tests, and identifies redundant test cases from large-scale test executions. Designed as a solo effort under Team SoloPath, it simulates execution, analyzes results using multiple intelligent agents, and sends dynamic reports via Gmail and Jira. By reducing manual triage, improving reliability, auto-healing and optimizing execution, it saves valuable QA time and enhances pipeline stability.
Detailed problem statement
Modern QA teams struggle with maintaining large and complex test suites, often dealing with flaky test cases, redundant executions, and undetected anomalies. These issues cause wasted resources, delayed releases, and decreased confidence in automated testing.
Manual identification of such problems is time-consuming and inconsistent. Our automation addresses this challenge by using UiPath agents to intelligently process test result data and automatically:
Detect anomalous patterns (e.g., timeouts, unexpected failures)
Identify redundant test cases that offer little value
Highlight flaky tests that behave inconsistently
The system not only analyzes data but also generates targeted remediation suggestions and distributes them via Gmail — helping testers take swift, informed action
Detailed solution
To solve the problem of test suite inefficiency and lack of visibility into test reliability, I designed and built a modular solution using UiPath Agent Builder.
The core idea was to process automated test execution data and run it through three specialized agents:
Anomaly Detection Agent
This agent analyzes each test execution for outliers in duration, error messages, and failure patterns.
It flags tests with abnormal behavior (e.g., timeouts, unrecognized failures), which are often symptoms of environmental or logic issues.
Redundancy Identification Agent
This component evaluates tests that show repetitive results across multiple runs.
It identifies low-value or redundant tests based on stability metrics like 100% pass/fail ratios and makes suggestions for optimization.
Flaky Test Detection Agent
This agent scans for inconsistency in pass/fail patterns and classifies tests as flaky.
It provides specific remediation recommendations like adding waits, isolating root causes, or reviewing selector logic.
Each agent outputs a structured summary (JSON), which is parsed and evaluated through gateways. If anomalies, redundancies, or flakiness are found, the system triggers the Gmail Connector to automatically send categorized reports to QA stakeholders.
The process is fully automated — from ingesting raw CSV test results to agent-based evaluation and report distribution — ensuring fast and consistent feedback to testers.
I used UiPath Studio for logic orchestration, Agent Builder for creating AI-driven agents, and Gmail for seamless communication, enabling testers to take immediate corrective actions without manual analysis.
Demo Video
Expected impact of this automation
The automation significantly reduces manual analysis effort in test management by automating anomaly detection, redundancy identification, and flaky test analysis. It eliminates the need for testers to sift through large volumes of test logs manually, leading to a time savings of up to 70% per release cycle.
This also enhances test quality by flagging unstable tests and suggesting remediation steps, improving test reliability and reducing false positives. As a result, teams can focus more on fixing real issues rather than chasing unreliable test failures.
ROI is realized through reduced QA cycle time, increased release confidence, and improved developer productivity. It also supports better compliance and reporting by automatically summarizing results and sending structured reports via Gmail integration.
UiPath products used (select up to 4 items)
UiPath Agent Builder
UiPath Assistant
UiPath Autopilot™
UiPath Integration Service
UiPath Maestro
UiPath Orchestrator
UiPath Studio
Integration with external technologies
Gmail
Jira
Sample inputs and outputs for solution execution
Inputs: A CSV file containing simulated test execution logs with columns: TestName, RunId, Status, ErrorMessage, Duration, TestType.
Outputs:
Anomaly Detection Agent: List of failed or abnormally long test runs.
Redundancy Agent: List of consistently passing or failing test cases.
Flaky Test Agent: Tests with inconsistent pass/fail behavior.
All outputs are compiled and emailed as reports using Gmail integration.
