I want to float a question that keeps coming up on Document Understanding projects, and I would rather hear how you handle it in production than assume my answer is right.
A process goes live. The training set looks fine. Then the client starts sending the documents that were never in that set — the ones that land in Validation Station and stay there. The model can be retrained. The delay is almost never the training job. It is the wait for a new labeled slice that actually matches what the client just sent.
So the practical question is this: If you could batch-produce photographed OCR samples for those incoming fail cases — with labels generated together with the images — would that speed up model updates and new document-type rollout? Or do you still refuse to retrain until real client pages have been annotated by hand?
What I am trying to reduce is not “more accuracy on a clean PDF.” It is how long the client spends seeing the same fail case after they have already reported it.
(The image below is the kind of sample I mean. Look at it as a training unit, not as a render: one document, one real capture condition, and the annotations that consumed a lot of time in training cycle.)
From my experience with UiPath DU projects, I wouldn’t wait for a large batch of manually annotated client documents before retraining.
What has worked better is using the real production fail case as the seed, labeling a few representative documents, and then creating realistic variations (mobile photos, skew, blur, lighting changes, stamps, etc.). That helps the model see the same document under different OCR conditions without waiting days or weeks for more client samples.
The training itself is usually quick. The slow part is getting good labeled examples. In most projects I’ve worked on, that’s the bottleneck—not the retraining job.
One thing I noticed is that synthetic data helps most when the issue is OCR robustness. If the client sends photographed invoices or low-quality scans, Validation Station volume can drop after adding those realistic variations. But if it’s a completely new vendor layout or a new document type, I still want real annotated pages because synthetic data can’t invent layout patterns it has never seen.
So I’d use synthetic data to speed up iteration, not replace real production data. A small set of real labeled failures + realistic synthetic variants has been a good balance for reducing repeat Validation Station cases.
@Alro_wilde I think synthetic data can definitely help here, but I wouldn’t completely replace real client documents with it.
If you already know the type of documents that are failing, generating variations with different angles, blur, lighting, image quality, etc. can be a good way to get additional training samples quickly instead of waiting for enough real cases.
One thing I would check first is whether the issue is actually with OCR/digitization or with extraction. If OCR itself is reading the document incorrectly, retraining the extractor with more samples may not solve the actual problem.
I would use synthetic documents to increase the dataset, then keep some real Validation Station documents for validation/testing. That way you can confirm whether the improvement actually works on production documents and not just on the generated samples.
Yes, that’s currently one of the more practical approaches.
One issue I keep running into is that OCR failures are often driven by deformation and lighting variation, which produce misdetections. If we generate data specifically for those edge cases and then rebalance the mix of styles in the training set, would that support faster production iteration. getting a new user template live in a few days with 90%+ accuracy?
@Alro_wilde Yes, that should help, especially if you already know the common failure patterns.
I would generate samples around those specific issues like skew, lighting, blur and different capture angles instead of generating a large amount of random synthetic data. Then mix those with real production samples while retraining.
Getting a new template live within a few days could be possible depending on the document complexity and volume, but I wouldn’t assume 90%+ accuracy just from synthetic data. I would still validate it against a separate set of real documents before moving it to production.
As in the picture above, I can already generate realistic lighting and geometric distortions.
Curious whether this kind of synthetic data can replace real samples, or if it still needs to be mixed with production documents.
@Alro_wilde Yes, it still applies even with a targeted synthetic set like this.
This type of data is useful because you are reproducing the actual conditions that cause OCR problems instead of just creating random variations. But I would still mix in some real production documents. Synthetic data can get very close, but there will always be things in real captures that are difficult to reproduce consistently.
Yeah mixed bag. What I can already spit out is just that targeted slice — same template, different lighting/geometry, labels generated right with the image. So we’re not sitting around waiting for another 30 client photos.
For people actually running this in prod: do you treat synthetic as extra coverage on the known fail cases, or are you also using it to stand up a new document type before the first real batch even shows up? Keep hearing different answers on that one.
@Alro_wilde I would use it for both, but in slightly different ways.
For known failure cases, synthetic data is useful to add coverage quickly without waiting for more production samples. For a new document type, I think it can also be used to build the initial model if the layout and fields are already known.
Once real documents start coming in, though, I would add those to the dataset and retrain based on the actual failures. So I wouldn’t wait for 30 real samples to get started, but I also wouldn’t rely on synthetic data alone long term.
@Alro_wilde
Interesting use case. I’d use synthetic data as an accelerator rather than a replacement for real production samples.
A good approach could be to take a few real failed documents and generate realistic variations around them: blur, rotation, mobile capture, lighting, compression, etc.
Then keep a small set of real client failures as a validation set. If the synthetic samples reduce Validation Station volume there, you know they are actually helping rather than just increasing the training set.
that split makes sense.
One thing that still bothers me on the “use a few real pages” side:
those pages are usually full of customer PII, so the template gets mosaiced or painted white before it can even enter the set.
After that the layout is already dirty - headers, stamps, table cells turn into blocks that production never actually sends. Doesn’t that make template discrimination worse rather than better?
agree it should stay an accelerator.
The catch is that the “real production samples” you would mix in often cannot stay real.
Privacy review mosaics or whites out names, IDs, account numbers, sometimes half the header. The model then trains on a contaminated template and I am not sure it still learns the document type - it may just fit the redaction artifacts.
