Has anyone measured DU/OCR lift from photorealistic “in-the-wild” docs vs clean templates + augmentation?

Help Document Understanding
, ,
1

Most Document Understanding / OCR training sets I see are still clean PDFs or flat scans. What actually hits production is a printed page on a desk or a phone photo: bad lighting, folds, glare, odd angle, tiny text.

Example
Example1500×874 222 KB

I’ve been generating that kind of “photographed in a real scene” page from scratch, with annotations, instead of template renders plus blur/noise/rotation.

Has anyone here measured whether this type of data actually improves DU results versus the usual clean set + augmentation? Even a rough number would help — digitization, extraction, or Validation Station volume.

Failure modes are just as useful. The cases I keep seeing are dense tables after a crease, small text at an angle, mixed-script pages, and glossy paper with window glare.

Curious if others ran into the same gap, or found a better way to close it.

2

Hi @Alro_wilde ,
Yes, this is a common gap in Document Understanding projects. Training only on clean PDFs often gives good lab results but struggles in production when documents come from mobile photos, scans, folds, glare, shadows, or skewed angles.

A better approach is to include real-world samples or synthetic photo-like documents in the training dataset. This usually improves OCR accuracy, field extraction, and classification performance, especially for:

  • Folded or creased documents
  • Mobile phone images with perspective distortion
  • Small text at an angle
  • Glare and uneven lighting
  • Mixed-language or mixed-script documents
  • Dense tables and handwritten annotations

In UiPath DU, the best results typically come from a hybrid dataset consisting of:

  • Clean PDFs (baseline training)
  • Real production samples
  • Augmented/synthetic photo-style documents

Also, use preprocessing steps such as deskewing, image enhancement, and contrast correction before sending documents to OCR engines.

Thanks

3

@Jasmin_Bar thanks. The clean PDF + rotate/fade mix is already common, so that part I’m not questioning.

What I haven’t seen measured is the extra lift from putting the page in a real scene ,versus that same document with only geometric/color augmentation. If anyone has a before/after on OCR or extraction for that specific gap, even a rough number, that’s the piece I care about.

4

Hi @Alro_wilde

I agree. Real-world images can be quite different from clean PDFs.

In production, things like folds, glare, shadows, perspective, small text and poor lighting can affect OCR and extraction.

It would be interesting to compare:

  • Clean training data
  • Clean data + augmentation
  • Realistic photographed documents

Metrics like extraction accuracy, validation rate and processing time could show how much realistic training data helps.

Would be great to hear if anyone has tested this in a real DU project.

5

@Monali_Vekariya hybrid set I agree. the blocker is cost — enough real phone photos with labels doesn’t scale.

so the open question is whether photoreal scene synthesis can cover that slice, with only a small real set for calibration.

6

@Alro_wilde

My proposed methodology to measure the effect is through an A/B controlled experiment design:
Condition A:

  • Clean documents augmented with geometrical and color transformations.
    Condition B:
  • Same documentation, augmented with photoreal “in the wild” synthesis.
  • Same model architecture and augmentation setup in terms of size of training set and other augmentation parameters.
  • Both the conditions evaluated against the same held-out test set of phones/photos with glare, folds, skew and small text.
  • OCR CER/WER and Field Extraction F1 reported, along with relative improvement of Condition B compared to Condition A.
7

A/B is right, but the comparison I care about is:

A: real photos + human labels

B: fully synthetic in-the-wild pages, same types, auto-labeled

Same model, same train size, same held-out real photos. Report CER/WER and field F1.

OCR/KIE is enough — VLM needs too much data.

8

That’s right, that analogy makes sense to me too. I would also ensure that the document contents/types and size of the training set match, so the only differing factor would be human-annotated photos vs machine-annotated photos.

Another piece of information that could be added to this paper is the labeling time/cost per 1K pages along with CER/WER and field F1.