How to remove Unicode characters while Extracting data from pdf

Help Studio
, ,
1

Hi All,

I am extracting data from a pdf using Document understanding, while i use Form Extractor it also extracts unicode characters(same as which are mentioned when you are creating template in form extractor, just below signature fields dropdowns), kindly help guys as the blocks are also getting extracted from the pdf and printing in excel as well

@ppr @supermanPunch @Palaniyappan @Gokul001 @postwick

Regards
Jai

2

Hi @Jai_Pande ,

Is it possible for you provide an Example String from the Extracted data so that we can validate it as well from our end.

Also, could check the below :

[^\u0000-\u007F]+

image
image1032×472 30.4 KB

We would require to perform a Regex Replace operation and remove the matched unicode values with empty values.

3

Hi @supermanPunch ,

Credit Card Application - Amily.pdf (990.0 KB)

4

It involves the Box Characters in Name Fields

5

image
image1848×225 37.9 KB

Its coming like this

6

@Jai_Pande ,

As suggested you could try with the approach with Regex Replace mentioned above and let us know if it works with the extracted data in Excel.

For the Extraction part, I do not think a more control could be provided to remove or Exclude the Square brackets as it is also included in between the strings. So maybe a post processing could be a better choice where you could perform the String/Regex manipulations to remove the unwanted characters.

7

@supermanPunch
Or can i use ML Extractor Endpoint also to just extract names?

8

@Jai_Pande ,

It depends on the Document type being used and the nature/template of the document.

Is the ML Extractor Model for the Form type already available, then you should be able to use it. Else a custom DU model would need to built (would not go there unless a careful analysis on the Document types/Formats/Templates are done and understood all the constraints and fields.

9

@supermanPunch
Also let me know that when i am extracting the boolean value from the form its showing in different lines, like i want one field and then it should choose between the two instead of it have to define separate fields like this

image

Want that i should only define Card Type field and in form extractor it should show me that what it choose

image

Its showing like this

image
image714×508 13.4 KB

10

@Jai_Pande ,

In the case when used with Form Extractor, I believe we would require to use separate fields for each value checkbox and then mark/label the Check for which the option is selected.

If for labelling in Document Manager where you are keeping the training data ready for DU ML Model, then you could use/define a Single Field for that Checkbox and select the Options instead of the Checkbox mark. But we should also make sure that we are training it with enough samples so that it would be having a balanced data on all sets of values.

Check the below docs :

11

@supermanPunch
Okay so if i am using labelling so then i have to only label the check box fields or all the fields as i am able to extract the data except for checkboxes

12

@supermanPunch

and how to select labelling getting a bit confused

13

request you to kindly arrange a small teams session @supermanPunch

thanks

14

@supermanPunch