Document Understanding - Pre-trained ML models for extracting data from ID Cards and Passports

Dear Community,

The 21.5 Cloud release of the Document Understanding AI capabilities brings some exciting updates. We have heard your requests for more pre-trained models, and this month we are launching with Passports and ID Cards.

Passports and ID Cards are often hard to read images with a lot of background image noise, designed to make them hard to fake. This makes them particularly difficult to read by many OCR engines, especially fields which involve a single letter, like the Sex or the Document Type Code (usually P for Passport) which ar particularly hard to spot. Our proprietary UiPath Document OCR usually does a good job reading these images. However, to make sure you are getting the best results, feel free to try out some other OCR engines and use the one which works best on your documents. Google Cloud Vision OCR and Microsoft Read OCR are also good options.

The Passports model was trained on documents from a wide variety of countries, including non European or non-Latin script based languages. Due to international conventions requiring that passports include English labels in addition to the local language, this model should perform well on most documents. The fields recognized by the Passports model are listed below:

  • passport-no‚Äč
  • last-name‚Äč (also called surname)
  • first-name‚Äč (also called given name)
  • nationality‚Äč
  • sex‚Äč
  • birth-date‚Äč
  • issued-date‚Äč
  • address‚Äč (mailing address of the person)
  • authority‚Äč (may also be the city where the passport was issued)
  • expiration-date‚Äč
  • mrz‚Äč (Machine Readable Zone at the bottom of main passport page)
  • state-code‚Äč (3 letter country code)
  • document-type‚Äč (Passport, Diplomatic Passport, Military Passport, etc.)
  • document-type-code (1-2 letter code)‚Äč
  • personal-id‚Äč

ID Cards are far more multilingual than Passports, and many countries have no requirement for including English language labels, so they will only be in the local language. This means the country coverage for this ML model is more limited than for the Passports.

Here are the countries supported in this first Public Preview release:

  • USA (all 50 states plus DC)
  • Canada
  • Australia
  • Romania
  • France
  • Germany
  • UK
  • Belgium
  • Netherlands
  • Austria
  • Spain
  • Italy
  • Cyprus
  • Finland
  • Hungary
  • Poland
  • Croatia
  • India
  • Switzerland
  • Hong Kong

The list of fields recognized by the model is:

  • document-type‚Äč
  • last-name‚Äč
  • first-name‚Äč
  • birth-place‚Äč
  • birth-date‚Äč
  • sex‚Äč
  • nationality‚Äč
  • id-no‚Äč
  • issued-date‚Äč
  • expiration-date‚Äč
  • issued-place‚Äč
  • authority‚Äč
  • id-no2‚Äč
  • mrz‚Äč (Machine Readable Zone)
  • address‚Äč (mailing address of the person)
  • state

Enjoy and please let us know how this works for you, so we can improve it based on your feedback!

The Document Understanding Team

1 Like


How can I use this?

Hi, do you have the schema file for Passports and ID Cards that I can import into Data Manager?


You can find them here: Use a Predefined Schema