Test Manager vs Orchestrator?

Help Test Cloud
,
1

We recently got our Test Manager license. We’ve been using Orchestrator to run our Test Cases/Test Sets to this point. I’m trying to figure out why we should use Test Manager vs Orchestrator and I’m struggling to understand the advantages and would like to see if anyone has some concrete examples of why they use Test Manager and what functionality they feel is beneficial vs using Orchestrator → Testing.

I’ve reviewed the documentation on Test Manager many times – below are my thoughts on the key benefits they list, but if anyone has some more insight on this, it would be appreciated. Just looking for some solid reasons why someone uses it or recommends it. Thanks!

Assigning automation from StudioPro to test cases:

  • This is done as easily from Studio to Orchestrator (using Publish Test Cases)

Assign test cases to requirements:

  • I understand this with manual tests, but with automated tests, this is usually part of the automated test anyway.

Dashboards to get a quick overview of your test results:

  • This is the only reason I can see to use this, but even then it’s very easy to see the results from Test Executions. Dashboards are nice, but does it justify the cost of the license?

Reporting:
- I don’t really see this being any different than Orchestrator. Any thoughts?

Manual testing
- I don’t see how this would be any different than having a spreadsheet or something similar related to the automated test. I understand that it’s in one place, but I can do that with pretty much any tool like ADO Test Plans.

Creating defects from test results:
- I can see the usefulness of this, but at the end of the day, I usually want to create a defect in my own words and I’m not sure I understand the benefit vs $$ for just this, given what I mentioned above.

A few other notes on this…

  1. In Test Manager, you don’t actually see the test running – just the result. In Orchestrator you do, and you can also view the logs as it’s running. If I have to use Orchestrator anyway to see the progress of a test, then why would I use Test Manager?

  2. Test Case ordering in a Test Set:
    This is still not possible in Test Manager or Orchestrator. This is more of a personal gripe, as they still haven’t added this and not sure they will. This is a basic QA requirement for a Test Set, and UiPath is the only test automation development tool that I’ve seen that doesn’t allow this. I won’t go too much into this as I’ve opened conversations about this on the forum before and have had a lot of conversations with UiPath about this. I just mention it because I’m referring to Test Management here.

2

Hello @dennisp

Hope you have gone through the below documentation. If not please go through it.
Test Manager provides multiple functionalities to handle the test cases that you are creating. You have the test queue which is different from normal queue. Also heer you can create the test set which helps to group the testcases based on modules and to execute.

Also instead of keeping manual testcases and requirements separately, it helps to create manual steps using task capture and to add directly in the testcases for better management.

Thanks

3

Hi Rahul - thanks for the reply. Yes, I did go thru that documentation several times and addressed the individual bullet points in my original post. Regarding the test queues - those can be created in Orchestrator as well. We are currently not using Task Capture, so that’s of no use to me at this time - it may in the future tho. You can also create a test set in Orchestrator, so I don’t see the advantage there. My post was to get some real world examples of how/why people are using Test Manager vs Orchestrator for different types of testing (Regression, Black/White box, etc.).

4

Hi Dennisp,

Reasons why our customers use Test Manager. These may or may not apply to your specific use case.

  1. Historical analysis of Test Execution results. In Orchestrator, if I am trying to compare one of thousands of test cases I executed yesterday to one I executed today, it would require a ton of navigation and scrolling. Test Manager allows me to view it easily.
  2. Test Planning. Test Manager provides a space to define/document what needs to be tested (this is similar to Automation Hub’s role in the RPA space). Without test planning, it becomes the wild, wild west with no consistency on how testing is getting done.
  3. Requirements Traceability. Is something being tested for the sake of checking a box OR is it driving value as aligned to a specific business requirement? This planning functionality is not available in Orchestrator.
  4. Perform Manual Test Execution.
  5. Create Defects. If something fails, I want to track that it failed, ensure screenshots and logs are available in my defect management solution, and assign it to someone so it gets fixed. Test Manager has native integration, Orchestrator does not.
  6. Integrate with 3rd party Test Management solutions. Test Management is critical to all Fortune 500 and Enterprise companies. Here is a good article that covers its importance - Importance of Test Management Tools in Software Testing Services- Testrig Technologies . Test Manager allows native integration with 40+ different tools, Orchestrator does not.

Hope this helps shed light on why UiPath customers are leveraging UiPath Test Manager!

1 Like
5

Thanks for the reply Vikram. You make some good points. #6 applies to me the most, and unfortunately, I haven’t been able to get that to work the way I want yet, but I’ll continue to work with support on that. :slight_smile: