New UiPath Document Understanding features have been released!

New UiPath Document Understanding features have been released!

Our new UiPath Document Understanding features have been released to Limited Availability in February 2020!

Note: The Receipt, Invoice, and Purchase Order ML Extractor are a Cloud Platform offering and will be released to general availability on April. Additionally, the on-prem solution will be also released in April.

Key Benefits

Flexible approach to extracting information from documents

You have the option to use templates with the Fixed Form (ex-Position-Based) Extractor, regular expressions with the Regex Extractor, or pre-trained models with the machine learning - ML Extractor. Also, you can bring your own OCR technology like Abbyy or Microsoft Forms to digitize documents before data extraction.

High accuracy for even ‘noisy’ documents

We are continuously improving our AI to handle the complexities found in real world documents that make them hard to read, such as noise, rotated and skewed documents, or low resolution from printing and scanning.

Drag-and-drop AI skills directly into your workflow in Studio

IntelligentOCR package (to be renamed to UiPath Document Understanding) allows you seamlessly drag-and-drop our new ML Extractor into the workflow - similarly like any other extractor. Same infrastructure, new supercharged brain.

How to Install in Studio

Note that you need to create an account on platform.uipath.com and use a License key from the Licenses page for these features to work. The key must be pasted in API-Key property of the document understanding activities.

In order to utilize the Document Understanding framework you must first install the UiPath.IntelligentOCR.Activities activity package:

  1. Open the ‘Manage Packages’ window
  2. Click ‘All Packages’ in the left navigation, then search for “UiPath.IntelligentOCR.Activities”
  3. Select the result, make sure the newest version is selected, then click the ‘Install’ button in the right pane

Next, you need to install the UiPath.DocumentUnderstanding.ML.Activities activity pack:

  1. Check ‘Include Prerelease’, then search for “UiPath.DocumentUnderstanding.ML.Activities”
    image

  2. Select the result and click the ‘Install’ button in the right panel

    Important - We must reiterate, make sure you check the ‘Include Prerelease’ checkbox.

The ML Extractor activity should be visible now in UiPath Studio, as shown below:

How to Use

Here are the demo videos on how to use UiPath Document Understanding:

UiPath Document Understanding Demo 1: Setting up the framework in Studio

UiPath Document Understanding Demo 2: Data extraction configuration

Additionally, see these step-by-step instructions to help you get started with the ML models:

  1. If you haven’t yet defined taxonomy for the documents you intend to process, you can do so using the Taxonomy Manager

  2. Drag in the Load Taxonomy activity and store the taxonomy in a variable

  3. Drag in the Digitize Document activity

    • Drag-and-drop an OCR Engine inside the Digitize Document activity
    • Define variables for both outputs: DocumentObjectModel and DocumentText
  4. Drag-and-drop the Classify Document Scope activity and once again drag and drop the keyword-based classifier activity inside of it to add a new classifier. Activate the classifiers to enable classification

  5. Drag in the Data Extraction Scope activity

    • Populate input variables DocumentObjectModel, DocumentText, Taxonomy and DocumentTypeId. The DocumentTypeId is a string you can see in TaxonomyManager if you click on the Document Type you need to extract.
    • Define variable for output: ExtractionResults
    • Drag in Machine Learning Extractor and drop it inside the Data Extraction Scope
    • Drag in Regex Extractor and drop it inside the Data Extraction Scope
    • Drag in Position-Based Extractor and drop it inside the Data Extraction Scope
    • Populate Endpoint input property with the URL of the endpoint you would like to use:
    • https://invoices.uipath.com for Invoice processing
    • https://receipts.uipath.com for Receipt processing
    • https://purchaseorders.uipath.com for Purchase Order processing
    • a local custom endpoint in case you have one of these models hosted on-premises
  6. Click the “Configure Extractors” link within the Data Extraction Scope activity

  7. Expand the Document Type you are interested in and populate, on the right side column, the names of the fields which correspond to the fields in your taxonomy. See below for the full field names you need to fill in on the right side. You can mix and match between values and what you want to extract.

Pre-trained ML model for Invoices extracts the following name and line item fields:

“name” 
“vendor-addr” 
“billing-name” 
“billing-addr” 
“shipping-addr” 
“invoice-no” 
“payment-terms” 
“due-date” 
“po-no” 
“date” 
“net-amount” 
“tax” 
“total” 
“currency” 
“line-no” 
“description” 
“item-po-no” 
“quantity” 
“unit-price” 
“line-amount” 

Pre-trained ML model for Receipts extracts the following name and line item fields:

“name” 
“total” 
“vendor-addr” 
“date” 
“phone” 
“currency” 
“expense-type” 
“description” 
“line-amount” 
“unit-price” 
“quantity” 

Pre-trained ML model for Purchase Orders extracts the following name and line item fields:

“po-number” 
“date” 
“client-name” 
“client-address” 
“vendor-name” 
“vendor-address” 
“shipping-name” 
“shipping-address” 
“billing-name” 
“billing-address” 
“payment-terms” 
“delivery-by-date” 
“net-amount” 
“tax-amount” 
“tax-rate” 
“total-amount” 
“line-number” 
“description” 
“product-code” 
“delivery-date” 
“unit-measure” 
“unit-price” 
“quantity” 
“line-net-amount” 
“line-tax-rate” 
“line-tax-amount” 
“line-amount” 
“currency”
12 Likes

So the ML extractors can be used to do extraction without templates, correct?
this can be done without AI fabrics and doesn’t count towards Machine Learning library use, right?

Yes. You can use our models in the cloud without AI Fabric.

AI fabric enables retraining and managing / deploying models generically, so it can be used with DU and other models.

The DU solution comes with everything you need to run DU on it’s own, so it can be purchased separately from AI Fabric.

However, using our models for DU does count towards ML use.

Hi - is it possible to re-train the models on-prem now? And hopefully the validation station can be completely skipped? thanks

is it support for Thai Character?

Hi,

I would like to know what Extractors can be used On-Premises and is there an alternative for using the Cloud based Extractors on Non Cloud environment.

One of the requirements we have doesnt have active internet connection and unable to connect to the cloud.

1 Like

Hello @chaithanyapallegar,

Regex based extractor and Keyword based classifier are usable with no internet connection.
Machine learning models can also be hosted on prem (so no external internet connection - except for the ML model infrastructure that needs to be able to do licensing calls only).
The rest are only available with an internet connection on the robot machines, but we have it as a WIP to make them available on prem as well.

Hope this helps,

Ioana

2 Likes

If I understand correctly you should not have a newer Studio version than Orchestrator version.

So do you need/recommend to have Orchestrator 2020.4 to use Document Understanding, or can you do it on 2019.10 LTS?

Can we use this for Structured as well as for semi structured Invoices?