Document Understanding: New Human-Robot Levels Available :)

You know Life’s a Journey, right? :railway_track:

So is Document Understanding: a journey of new features and capabilities, of constant improvements, and, of course, of your files through your processes! :hiking_boot: :mountain:

To “help you pack :world_map:”, we’ve prepared some nice little artifacts for you:

:sparkler: Public OCR Contracts :tada:

Remember the open and extensible framework we’re building?

Keeping true to our promise, we are now publishing a Public OCR Contract nupkg, so that YOU can build Custom OCR Activities!

If you are using an OCR engine or service very specific to your needs, you can now wrap it up in a custom activity and make it available for usage in both Screen Scraping and UiAutomation activities, as well as in Document Understanding.

Through this new UiPath.OCR.Contracts version, we are opening the UiPath ecosystem of activities to any and all customers, partners and technology partners, to publish their own OCR engines, tailored to the markets and use cases they face.

:question: How to Use This

Use the latest preview version of the UiPath.OCR.Contracts package in your Visual Studio project, and start building.

The documentation around the public contracts is available here. Make sure to check out the code samples in the documentation, and you’ll get to…

:arrow_down:

image

:arrow_down:

And the journey continues!

:sparkler: Classification Station in Action Center

You’ve probably seen it already as an attended activity. Now, you can take it to the next level, and have your Long Running Processes take a well-deserved “break” while the humans get the chance to look over the Automatic Document Classification results.

( For those of you who missed our previous announcements, see this to get a full context around the new document classification, splitting and validation capabilities)

:question: How to Use This

Upgrade your workflows to the latest release of the IntelligentOCR package (tick the Include Pre-Release flag please :bowing_woman:), and use the Create Document Classification Action and Wait for Document Classification Action and Resume activities:

image

Just like for the Document Validation activities (about which you can find out more over here), the Classification Validation activities use the Orchestrator’s Storage Buckets for storage support for the lifetime of the human task, and communicate from one to another through the ActionObject output:

:arrow_down:

image

:arrow_down:

:arrow_down:

image

:arrow_down:

(see the trail? :smile: and it continues…)

:sparkler: :heart: Extractor to Trainer Communication + Machine Learning Extractor Trainer :tada:

So now that classification is done, the Data Extraction part comes into play.

You are probably already using the Data Extraction Scope activity in your workflows. Now it’s time to enable the Train Extractors Scope as well, to enable the feedback loop for ML Models!

:question: How to Use This

Straight forward: after the human validation step (be it as an attended activity or integrated into Action Center), add a Train Extractors Scope. Within it, add the newly released :tada: Machine Learning Extractor Trainer :tada: activity.

The Machine Learning Extractor Trainer will collect the human feedback for you, in a directory of your choice. Once you collect data and you want to retrain your model, just zip the content of the directory and upload it in your Data Manager for curation. Now that you know what your machine learning model will learn (after review and potential corrections), just export your new dataset, and start a new training pipeline for your ML Model. Whenever you’re satisfied with the performance of the new version of your model (performs better than the current one), you can then promote it to Production in your AI Fabric instance.

:arrow_down:

image

:arrow_down:

:arrow_down:

:arrow_down:

:arrow_down:

And you’re done :clap:

… well, almost.

Another thing that we’ve added, is the possibility for an Extractor activity to communicate :loudspeaker: with a Trainer activity. In order to enable this, you will need to “get them acquainted” :handshake:. The way to do this, is to use the Framework Alias boxes that appear next to each extractor and each trainer in the Configure Extractors wizards of the Data Extraction Scope and Train Extractors Scope.

Tips and Tricks :mage:

If you do not already have an ML Model that you want to retrain, but just want to collect data for a future model, you can do this by clicking “Cancel” when the Machine Learning Extractor Trainer’s wizard pops up. This will allow you to manually enter, in the Configure Extractors Wizard, the field names that you want your future model to be trained on. Not entering anything (just ticking the box) in this case would generate field names equal to the field IDs in the Taxonomy, so you might want to use prettier ones… :cherry_blossom:

If you have multiple robots executing these processes :desktop_computer: :desktop_computer: :desktop_computer: , make sure you collect the data from all of them… the more the merrier :smile: they say :slight_smile:

If you are capturing data for training using the new Machine Learning Extractor Trainer activity for a model that you are also using for prediction in the Data Extraction Scope, then please don’t forget to:

  • give the Machine Learning Extractor an Alias (any string would work)
  • give the Machine Learning Extractor Trainer the SAME Alias

So that the two components know they are paired. Use the Configure Extractors wizard to set these aliases.

:sparkler: … and We Keep Going!

While we keep treading the path towards getting the Document Understanding Framework as complete and robust as possible, do lend us a helping hand by sending us feedback on what you love :heart_eyes: or hate :angry: about it, what you feel is missing :ghost: :face_with_monocle: or not working properly :woozy_face:

Until next time, safe journey to all!

The Document Understanding Team.

49 Likes

@Ioana_Gligan
OMG 2 releases news on one day :partying_face: :rocket: :sunglasses:
what a great news :star_struck: :star_struck: :smiling_face_with_three_hearts: :smiling_face_with_three_hearts:

2 Likes

Great news on trout, I’ll say :smiley:

UiPath Rocks :beers: :beers: :partying_face:

3 Likes

Aah…When I feel I m done…
You guys make me feel Hey man you are not yet…!

Simply amazing
@Ioana_Gligan

10 Likes

Looks great ,document classification on action and train AI model in validation station

1 Like

This is the feature to die for. Feels like we have now a complete end-end solution for DU. So excited and can’t wait to get my hands dirty.

1 Like

Wow… It’s a Great news :slight_smile:

1 Like

Looking very very good! Excellents work to the UiPath team !!

1 Like

Great :+1:

1 Like

Is this entire thing available in community edition?
what are the limitations compared with a licence version?

1 Like

Great enhancements to the existing Document Understanding.

1 Like

Limitation for community:

  • you have page limits (max 1 or 2 pages depending on used extraction methods)
  • you cannot train your own custom ML model.
4 Likes

Great the Train Extraction Scope i was looking for .

2 Likes

Awsome :+1: :+1: :+1: :+1: :+1: great news

2 Likes

Thanks for sharing

2 Likes

Thanks

1 Like

Hello, @Ioana_Gligan

The Machine Learning Extractor Trainer sounds like a very cool feature and I would really like to test how it works in practice.
However, I got stuck at the Data Manager part:

just zip the content of the directory and upload it in your Data Manager for curation

I am note sure which are (or where to find) the registry credentials (username, password) mentioned in the docker login command from the documentation: AI Center

I am using an Enterprise Cloud trial license. Is there something I am missing?
Thanks.

Yep, that’s another type of licence you need to aquire via Uipath sales.

You need to reach out to your UiPath contact to get one - OR, if you can wait for another few weeks, you will see DataManager available directly in AIFabric Cloud :slight_smile:

1 Like

I was really hoping to use DU and ML to use with excel documents. Do you see that being included in a future release. Or can you use ML outside of DU?
Thanks