How would you improve ReFramework?

Help Activities
, ,
1

Hello all,

In your opinion, what would you do different in order to make ReFramework better?
What are the flaws of Reframework that you find bad?

2

Not really a case of making it better but would create the following:

Datatable Based RE-Fremework
Files based RE-Fremework
Email based RE-Fremework
Database based RE-Fremework
Data Services based RE-Fremework

For me these components are the most used when it comes to automating projects

3

The Config managing assets is the worst thing, especially because its in a dictionary, and something I have removed entirely.
Standard settings such as the number of times to restart an application etc I have put in a custom object. For assets to hold things used in the process like a website URL or credentials, I just use the asset directly where I need it, so if I need it in ‘Applications Start’ I just add the asset activity there.

This massively simplifies your code and tracing down assets, allows the Orchestrator to validate if an asset exists, and reduces the risk of someone messing up with passing around a credential across the entire process.

After that, an easy one is the layout of workflows. I hate that in the REFramework you have to go to the ‘Framework’ folder and ‘just know’ which workflows are ones you should be changing and which ones contain core functionality that shouldnt be changed, this is simple to fix with some basic folder grouping.

Thirdly, the error handling isn’t robust enough, it needs to log full exceptions better and handle cascading errors better.
It also doesn’t allow you to easily switch out the email handler you are using to communicate any errors nor allow you to easily configure where any error screenshots take go so you can, for example, add them to sharepoint instead of to the local robot disk.
Related to this, it doesnt have a good setup for using the Integration Service for any of these things.

There are many more, but I built my transactional processing template from scratch because the REFramework wasnt robust enough for me and I have functionality for all the above.

1 Like
4

If you change the design so that the ‘Get Next Transaction’ is truly modular and abstract you don’t need 5 different versions of the REFramework, you just have a single workflow that is super easy to change to get the next transaction from a data table, or folder, or mailbox, or database etc and, in doing so, you allow it to handle even more scenarios such as working down a table in a web app directly.

1 Like
5

For context Etienne, if you see here, the logic for deciding IF I should get another transaction (so checking stop request, how many errors etc) is handled elsewhere as part of ‘core functionality’.

This workflow (which is grouped with my easy to find ‘changeable ones’) Just expects you to get a queue item somehow. Mostly simply this is from a Queue, and you configure that directly and simply straight in the workflow (no config nonsense).
If you want to use emails from a mailbox for example, no problem, add some code to get the next mail, and make it into a queue item using the ‘Add Transaction Item’ activity which makes the queue item but also starts it and you are ready to go. Easy implementation that still leverages a queue for reporting, but lets you easily connect it to a mailbox.

The image shows an RPA development environment with a workflow in progress for "Get Transaction Item" within the Performer Framework, displaying dependencies, project hierarchy, and variable arguments. (Captioned by AI)
The image shows an RPA development environment with a workflow in progress for "Get Transaction Item" within the Performer Framework, displaying dependencies, project hierarchy, and variable arguments. (Captioned by AI)1405×830 57.3 KB

6

My biggest gripe with the RE Framework is the Excel config file. I disliked it so much that I created my own LoadConfig process using a bunch of YAML files. Basically you split the config into a bunch of YAML files based on the kind of config they hold; they get loaded through a main YAML file. So, I have YAML files for Email, Orchestrator, Asset (incl cred asset), Project, API etc etc. The RE Framework Excel config wouldn’t take a credential asset. With YAML based config, you can keep them open while the bot is running/testing.

The next issue is the time it takes to set up an RE Framework just to get the basics working.

There are others, but kind of easy to get around.

7

I see what you mean. That will make it allot easier.
Thanks

8

That is an interesting approach. I like it.

9

All of that is solved if you nuke the config entirely and just use an asset directly where you need it. There are a ton of over engineered solutions, such as converting to JSON or YAML and having 3 configs for different environments. Its all solved by just liberating yourself from the yolk of the config dictionary and removing it altogether :slight_smile:

10

I don’t use a Config Dictionary :grinning:

I used to do that initially, but managing 30-50 xaml files per project and having assets scattered everywhere was too much management!

11

I’d argue that you have too much in a single project so thats a different issue, but how is it easier with a YAML?

You can find all assets in a couple of seconds by using the universal search and if you want an overview you can easily see it in the Orchestrator when you publish the package.
I dont see how the YAML makes it ‘easier’ (although like I say, you should probably already break the process apart if you have so many workflows).

12

Just playing the devils advocate here (just because I can).

An advantage of the config dictionary is that it is one bundled object that contains all external config data. (exception here for credentials, they should have a minimal lifespan)

Accessing assets directly from code, makes these references volatile to changes. If for whatever reason an asset requires a name change (shouldn’t but I’ve seen weirder things). You’d have to indeed search through the entire project, making possibly multiple changes risking mistakes a lot more than changing the mapping in the init once and done.

It touches the same thought pattern of libraries; why build something twice if once will do.

13

Hi @bp777

Some of the first things I would do are:

Initialisation State:

  • Add Init Retry 3 times
  • Log InnerException message “Exception.InnerException.Message.ToString” (within the Catch block). Can give more detail sometimes.
  • Add Screenshots (within the Catch block).

Process State:
Within SetTransactionStatus.xaml

  • Log InnerException message “Exception.InnerException.Message.ToString”
  • Log the Exception Type (“Exception.GetType.ToString”)

End Process State

  • Log InnerException message “Exception.InnerException.Message.ToString” (within the Catch block). Can give more detail sometimes.
  • Add Screenshots (within the Catch block).
  • Notify (via email) if the jobs faults.

New Out Arguments

  • CountOf_Success (Int32)
  • CountOf_BusinessException (Int32)
  • CountOf_SystemException (Int32)

These arguments are all incremented accordingly in the Process State before/after SetTransactionStatus.xaml.
After each job you can quickly view how a job performed by clicking the details icon. image

Note - screenshots at Init and End Process are just suggestions.

These are the first things I would do to improve the standard REF template (there are more things).

Cheers

Steve

14

Except, that mapping out your assets from the config to actually where they are used in code is far more intensive. Tell me how do you find where it is used in your project? By searching the name of the asset? Ok, but then thats exactly the same as if you use your asset. As I explained a universal search can easily find every asset activity to make this overview in seconds. Plus you can much quicker figure out if an asset is used, in a ‘config dictionary’ project if I say to you to remove the unused assets its a royal pain to figure out if any are actually used or not.

Furthermore, as I explained, when you publish your package, the Orchestrator will validate if the assets you used are available, which is nice to catch missing assets as you move between dev > Test > Prod. A config dictionary prevents this functionality.

I know people have a strong connection to the dictionary, but there is really no argument that holds up, especially if you already manage some assets (credentials) by directly using them, the hybrid approach already undermines the foundational argument (which is bad) of using a dictionary.

15

I hate seeing this.
Log the entire exception by serializing it. Why?
Well firstly, not every exception has an inner, meaning you either cause another error or need even more logic in the catch block, but also because the exception contains alot more than the message. It often contains the specific workflow and activity name that faulted, the data dictionary may contain useful debugging information, and its wrong to assume that an exception only has 1 inner exception, that exception might have another.

If you are going to log it, log it fully and use all the information given to you instead of throwing most away.

16

Yes that’s true, you do need to try catch the inner exception which adds to the busy-ness within the catch block.

However it can direct your efforts when the exception message is not helpful.

The “Exception.Source” is helpful to log also which usually logs the activity name itself or the corresponding dependency.

What else do you like to log?

17

I log the entire exception by serializing it. Its simple and effective and basically always works. I have not found an exception that cannot be serialized so far and its a bad exception if thats the case.

18

I second the post that already point this out, not that it’ll make it “better” but i was thinking about different reframework base on the type of data you want to handle

19

It is a bit difficult to explain everything without showing you how my config works, but I don’t want to do it yet :grinning:

  • YAML file has less clutter compared to, say JSON and it is possible to define variable types.
  • As explained before, unlike the Excel config, having the config files open doesn’t crash the app.
  • I have a need to predefine the “variable” names(like API_User, API_Password, ERP_User, ERP_Password, DB_User, DB_Password, WEB_User, WEB_Password etc etc) produced when running the config; a result of having multiple credentials stored and retrieved from assets. This helps me with staging - dev, savvy user testing and production.
  • When sharing my xaml files, I don’t have to “clean” them, as by default there’s no identifiable information included in them as the receiver wouldn’t even know I am using a stored credential asset. I only need to protect my config files, which I can even produce dynamically from a database, if I choose, for production.
  • The app will not crash if I use the wrong case (upper/lower) - again unlike the dictionary config
  • Even though the difference is not huge, generally, reading text files is faster than reading Excel.
  • I have the freedom to mix and match different config files to make up a full config library; this happens dynamically. Also hierarchical branching is possible, e.g Config.AsString(“SFTP/Production/Host”)
20

I very much get how a YAML works, and I’ve seen it. I’ve seen the Excel approach, the JSON approach, the YAML approach. I am well versed in the ‘options’.

To address some points, you mentioned the Excel crashing the app if open, thats only if you use the classic activities. Modern Excel activities (which have been around for like 4-5 years now) don’t have this issue. Not saying you should use Excel, but its not a good point against it.

You mention all the arguments you need to give prefixes to. I don’t see how the YAML helps really. Again, just use the assets directly where you need them, having a smaller scope reduces the number you pass around and simplifies everything and there is no issue moving between tenants, the assets just need to match name and the Orch helps highlight any missing ones, unlike a YAML approach.

I am not sure what you mean by needing to ‘clean’ a workflow. Perhaps you can explain that further, but I don’t understand what security you are implying exists from using a YAML to getting an asset. I’d be just as easily able to read your assets in the YAML than an asset in a workflow?