Document Understanding: Machine Learning Document Classification Community Release

Greetings,

The UiPath Document Understanding team is pleased to announce the release of much awaited Machine Learning Document Classification functionality.

What is Machine Learning Document Classification?

Machine Learning Document Classification functionality is a suite of capabilities that will help users classify documents using a custom trained ML model. This will augment current classifier offerings such as Keyword Classifier and Intelligent Keyword Classifier. As a part of this offering, we are releasing three product features – 2 activities (Machine Learning Classifier - MLC and Machine Learning Classifier Trainer - MLCT) and 1 Out-of-Box (OOB) ML package (DocumentClassifier).

When to use Machine Learning Document Classification?

Machine Learning Document Classification can be used in situations where the other simpler classification techniques such as Intelligent Keyword Classifier might not provide accurate results. While this technique can be used on any document set of reasonably big size, it is more preferrable for scenarios where you have high diversity in document sets.

How to use Machine Learning Document Classification?

Let us say, you want to classify a document into four classes – receipts, invoices, purchase_orders and utility_bills. You can do this using Machine Learning Document Classification in three easy steps:

Step 1: Creation of a labeled dataset for ML model training

Follow these sub-steps:

  • Create a project in AI Fabric and an empty dataset in the project like this:

    image

  • On your Studio workflow add “Machine Learning Classifier Trainer” in the “Train Classifiers Scope”. When you refresh the “Project” and “Dataset” fields, you will see a drop-down with possibly multiple entries, including the project and dataset that you create above:

    image

  • Configure the “Machine Learning Classifier Trainer” using “Configure Classifiers” in the bottom. If you are creating the ML model for the first time (as opposed to trying to create dataset to revise an existing ML model), skip the part where it asks for “ML Skill” and just proceed to next screen and manually enter the names of classification classes and map them to the Document Types declared in the Taxonomy Manager, as shown below:

    image

  • If you have an existing ML Skill and want to create more labeled dataset to improve the model, just use the skill to find the class names. In this scenario, you can do the mapping between classification classes and Document Types specified in the taxonomy by first using “Get Capabilities” and then just selecting the correct class names from the drop down:

    image

  • As you pass document through the workflow containing the Machine Learning Classifier Trainer, the document will get labeled and stored in appropriate folder on AI Fabric. You will see a structure like this:

    image

    image

Now, we have a labeled dataset created and can move to the next step.

Step 2: Create a ML Skill for Document Classification

Follow these simple steps to create the ML skill (If you have never used Document Understanding packages from AI Fabric, please review more detailed directions here: About ML Packages):

  • Create a package using “DocumentClassifier” package available under Out-of-the-box-Packages in UiPath Document Understanding

  • Once you create a package, you will be able to go to “ML Packages” and can see the package you just created:

    image

  • Create a train pipeline using the package created above and the dataset you created in Step 1:

    image

  • Create a ML Skill using the trained package:

    image

Congratulations, you have successfully created a ML model that can now be used in a workflow for document classification!!!

Step 3: Perform Document Classification Using the Custom ML Model

Follow these simple steps:

  • In your workflow, drop the Machine Learning Classifier and point it to the ML Skill created in Step 2. You will also need to provide Document Understanding API Key as shown below:

    image

  • Next, click on Configure Classifiers to specify the type of classifier you want to use for a specific document type and to match “Document Type” names used in Taxonomy to ones used by the ML Skill. The later can be done very easily by using “Get Capabilities” by either configuring it when you first dropped the activity or clicking on the gear icon and then simply selecting the class names from the drop-down

    image

  • Everything is all set, test your Document Classification model:

    image

21 Likes

This is super amazing!!

Just tried it out with one of my models, and it worked very much better than the intelligent keyword classifier…

I trained the model using some sample documents that I usually use to train, and applied it on the same set, which gave the accurate result. I then used it against a totally new set, and out of all documents, I got the correct result for about 80% of it. There were few however that didn’t get identified, but with another round of training, things started falling in line :slight_smile:

Awesome stuff!!

edit:----
did a small video on this feature. Enjoy!

11 Likes

Thanks for the feedback, Lahiru. Loved the video! Amazing stuff!!!

3 Likes

As far as I understood, the Machine Learning Classifier Trainer sends the document text to the dataset, and then this text is used to train the package, similar to how the Intelligent Keyword Classifier recognizes words unique to each document type. So what are the advantages of using one classifier over the other?

1 Like

Hi @pedro.cavadas

Good question. The main point here is the accuracy. With my experience on document understanding, I have noticed that Intelligent Keyword Classifier (IKC) sometimes doesn’t provide the expected classification, or sometimes gives it with lesser confidence level, so that we are forced to use manual classification for such documents. This mainly happen when you have a high diversity across all document types.

For example: i used to process purchase orders and couple of other types of documents. But purchase orders we processes had a very high diversity… we used to get huge documents with multiple POs in it… and each PO had lot of variations… sometimes it is difficult to understand whether its a PO by just looking at it.

So, in this case IKC might not give a satisfying result. But I used the ML Classifier and trained it with a bunch of documents…When applied, it was able to give a better result… and overtime with multiple training attempts it can get even better…

1 Like

The newly launched – Intelligent Document Classifier 1.0 allows users to gain hidden insights by classifying documents, based on structural features and/or textual features. It uses the new-age technologies, machine learning (ML) and artificial intelligence (AI), to enable layout- and content-based document classification.

Text Classification Services

MeaningCloud: MeaningCloud is a set of APIs (application programming interfaces) for text analytics, including text classification

The MonkeyLearn platform can be used to build a custom text classification tool to categorize your text data as per your programmed specifications

Google Cloud NLP: If your data is already stored on Google’s cloud, their NLP service may be an easy way to smoothly transition into text analysis.

IBM Watson: The Watson Natural Language Classifier is part of a suite of text analysis tools available with IBM Watson.

Aylien: Specializing in the analysis of news articles, Aylien’s text analysis allows you to create a custom text classification model without leaving your browser.

Rosette: Part of Basis Technology, Rosette’s text classification system comes pre-trained on the IAB Tech Lab Content Taxonomy, but can also be customized through keyword-based training or a training dataset. To know more about ML visit @CETPA

Hi @Lahiru.Fernando and @Tarun_Singh ,

Can Machine Learning Classification be able to split document the same as intelligent keyword classifier ?

1 Like

Hello @Thong_Mai_Tr_ng_Hoang

Nope. Machine Learning classifier is used to classify large varying documents that belong to same Classification.

However it does not classify pages in the same document.

I had this requirement on one of my recent processes. I used the ML classifier due to the nature of the document, but I also had the requirement of splitting it into multiple documents. So I combined the ML classifier with Intelligent Keyword classifier to do both at the same time :slight_smile:

1 Like

Hi @Lahiru.Fernando
Do you mean to say that if there is any merged document of 4 pages containing invoices, electricity bills, property tax and caste certificate on each page then it will classify only the document which has got higher confidence score and it will not give a classification array like intelligent classifier for each page?

We are experiencing the same problem with ML classifier…on a merged doc …it is classifying only one doc type in the above example.

Our ML classifier model is made for these doc types but if this does not work on merged docs then what is the use of making ML classifier?

Please reply.

1 Like

@Lahiru.Fernando Please reply.

1 Like

Hi @vibhor11

Yes… Let me explain as follows.
As you know, Basic and Intelligent Keyword Classifiers rely on keywords to identify a document type. However, there can be scenarios where documents belonging to a specific category are not very similar and vary in context. In such scenarios, it is challenging to identify documents using keywords. This is where the Machine Learning Classifier can help more. However, ML classifier cannot split documents.

Here are the capabilities:
When to use Keyword Classifier

  • your files contain one and only one document type each (so no file splitting is required);
  • your files contain evidence related to the document type in the first three pages of the file.

When to use Intelligent Keyword Classifier

  • your files contain one or more document types within a single file
  • your document types are relatively easy to differentiate as far as content goes.

When to use Machine Learning Classifier

  • Your need to classify the single documents into different document types. No splitting is required.
  • The custom document types are very similar. A trained Machine Learning Classifier can differentiate more easily between two similar document types than the Intelligent Keyword Classifier.

So, the purpose of the ML classifier is different than splitting. To split documents, you must go with the Intelligent Keyword Classifier.

Refer to:

1 Like

Thanks @Lahiru.Fernando, This is so informative and cleared many doubts in queue. Thanks again.

1 Like

Thanks! @Lahiru.Fernando
By when we can expect this splitting functionality in UiPath ML classifier?

I will check on this and update you once I have info…

Awesome… Feel free to connect whenever you need… Happy to help anytime.