Generative Extraction & Classification using Document Understanding in cross-platform projects - Public Preview

[Monica_Secelean (Monica Luca)]

Overview

Note: As with all preview features in Studio Web, this functionality is only available for Community accounts and not accessible for Enterprise - to try it out without a community account, feel free to download the DocumentUnderstanding.Activities 2.3.1-preview package in Studio.

We understand that classifying and extracting information from diverse document types can be challenging and time-consuming, especially when dealing with custom or unstructured use cases. Therefore, we are excited to introduce generative capabilities in our Document Understanding Activities package - having the Classify Document and Extract Document Data Activities now empower users to input prompts based on which to either classify the document or extract data from it, making the process more intuitive, efficient, and flexible!

In this way, when working with either activity, users have the option to select the Predefined Project and the “Generative Classifier” or “Generative Extractor” as model to work with - requiring the input of key-value pairs, where the user can provide his prompt as input as sampled below:

The prompt will then be sent to a large language model, together with data from the document, to classify or extract the required information which will then be consumed in the workflow.

Note: At this time the generative model is not retrainable and as always, all data handling adheres to our standard terms of service.

Documentation

Activities - Activities

How to Get Started

Pre-requisite: DocumentUnderstanding.Activities - min. version 2.3 package
Simply create your cross-platform workflow in your preferred Studio environment and when using either Classify Document or Extract Document Data:

1. Select the Predefined project
2. Select the Generative Classifier or Extractor
3. Provide your prompt as key value pairs, where:
   • key will be the Document Type (e.g. CV) or Field Name (e.g. email)
   • value will be a description for determining either of them (e.g., CV containing candidate skills and experience or email address of the document)
     When running your workflow if using the Validation Station, one can see why the extractor has selected a particular answer for a field.

Limitations

Table processing may not always lead to the best results - we’re working on fixing this, so if you encounter issues, please shout out

Charging

Charging will happen based on AI Units - we don’t have this finalized yet, but we will update you here, once we have all details in place.

Do reach out if you give Generative Classification or Extraction capabilities a try and let us know how it’s going! What are we missing? What would you like to see further? Looking forward to your thoughts!

[Joshua_Allan_0 (Joshua Allan)]

Having followed the ‘How to Get Started’ instructions the generative extractor doesn’t appear to be there for us.

We are on the latest version of Document Understanding activity available too.

Can you provide any guidance on how we can get access to this?

[Monica_Secelean (Monica Luca)]

@Joshua_Allan_0 the feature is part of the 2.3 activities package - I do not see it in your list of packages, can it be that you didn’t check the “include prereleases” checkbox? So that you can also see packages which are not GA, but preview

[Joshua_Allan_0 (Joshua Allan)]

Ahh problem solved. Thanks!

[ebeloglavec (Emil Beloglavec)]

I wonder if this activity is also available in Studio Web?
Kind regards,
Emil

[Monica_Secelean (Monica Luca)]

@ebeloglavec yes, absolutely!

[islam.spaho]

@Monica_Secelean - In the Extract Document Data activity, I don’t have the option to select from the drop down. It’s basically empty. Anything I might have forgotten?

[Monica_Secelean (Monica Luca)]

@Monica_Secelean

the feature is part of the 2.3 activities package

can you make sure you have this installed and let us know if it’s still not working? I’ll modify the forum post in the meantime

[islam.spaho]

Still not working

[Monica_Secelean (Monica Luca)]

@islam.spaho what Studio version are you using?

[ebeloglavec (Emil Beloglavec)]

Is there a setting that must be adjusted because we do not see Generative Extractor in Studio Web?

Thank you very much for your help.
Emil

[islam.spaho]

i’m using Studio 2022.10.3

[Monica_Secelean (Monica Luca)]

@ebeloglavec is this an older workflow of yours? ideally, for new workflows, it should be there (as they would reference our latest package version)

[Monica_Secelean (Monica Luca)]

@islam.spaho looks can you retrieve any projects in EDD? if not, are you connected to your orchestrator instance?

[ebeloglavec (Emil Beloglavec)]

Even in a new workflow the Extractor does not show up.

[balaraman.ramiya]

@Monica_Secelean

Note: The data sent to the Generative Extractor/Classifier will be sent to an LLM Model instance which is not publicly available, will not leave it and once processed, it will not be stored or used for training.

Hello Monica,
This mean the model (Trained by UiPath) and managed by UiPath?
Regards,
Balram.

[ryan.brown]

Very cool, working on my end!

[islam.spaho]

I’m connected to Orchstrator and the dropdown is empty. No other Projects to be selected. The funny part of the story is that i can select a project in Classify Document activity and not in EDD. ^^

[Monica_Secelean (Monica Luca)]

@ebeloglavec any chance you can export the wf and send it to me at monica.secelean@uipath.com?