CV localserver: handling 800 word limit

[Whynotrobot]

While building something, I found out there’s a limit in ComputerVision Localserver. It can only handle screens containing less than 800 words.
It’s inconvenient that this limit is not in the documentation btw. And of course the limitation itself is inconvenient.

Are there ways to circumvent this limitation? My application can have a large datatable on display, triggering this 800 word limit.
Since the contents of the table are not important for my workflow, I selected a part of the application in the CV Screen Scope. So when indicating the application, I draw a rectangle around it, but only the upper part.

This basically adds a limited ClippingRegion to the Target, besides the selector. And thus limiting the amount of words visible.
But of course, this may not be very robust.

Are there other ways to handle this?
Can the CV-activities for example be set to ignore everything after the 800 words, to be able to continue processing?

[Bogdan_Sultana]

Hi @Whynotrobot,

First of all, thank you for posting, it helps a lot with fixing/improving the product.

@Whynotrobot

It’s inconvenient that this limit is not in the documentation btw.

You’re absolutely right, we’ll add this as soon as possible, along with some more details around why we output this error.

@Whynotrobot

And of course the limitation itself is inconvenient.

The main issue with hitting this limit (maximum number of words the Screen OCR can detect) is that this (design-time) output is a random subset of the entire set of words on the screen - this is an unstable state, as we cannot guarantee the same output will be detected at run-time. This is why we output this error, to prevent the users from using an unstable state of the OCR detection.

We recently discovered this issue and introduced this limit - the 21.10 Localserver doesn’t have this error, so you can test and see if it’s stable enough (make sure to test multiple run-time scenarios) to do the job.

Simply raising the limit is an option, but it affects processing time (this limit is used while scanning for text boxes at different scales - the more you search for, the more time you spend in processing, obviously); for now, we think it’s safe to raise it to 2000 words without a serious impact on processing time (we’ll do this in the 22.12 Localserver release). As the processing for Localserver is done on the local CPU, we’re also thinking to parametrize this limit - but we’d like to avoid adding parameters for something that should just work.

Your approach (using a clipping region when indicating) is the right one - you’re saying “this may not be very robust” and it’s true, in the sense that the clipping region will always be used at run-time, so in case this region is not well defined given the run-time state, the automation might fail. But as long as the indicated region will be stable at run-time as well, this should work as well.

Alternatively, you can also disable OCR processing from Project Settings > Computer Vision > CV Methods:

if your automation can be designed and run using UI Elements alone. While testing this, we found a bug in Localserver 22.7 - it seems the Localserver ignores this setting, so we’re fixing it in the (soon) upcoming 22.12 Localserver release.

@Whynotrobot

Can the CV-activities for example be set to ignore everything after the 800 words, to be able to continue processing?

To answer your question, yes, but it’s not a good idea, since this is really a random sub-set of the OCR boxes on the screen. You can test using Localserver 21.10 and see for yourself (maybe you’ll get lucky and you’ll end up with the same subset at both design-time and run-time, who knows? ).

So, in short, watch out for the Localserver Release Notes, we’re targeting Jan, 16th to release the 22.12 Localserver, which will have a 2000 words limit + the bugfix for disabling OCR in CV Methods.

[LATER EDIT]
Current release target date has moved to mid-april (we wanted to include a bug-fix that needed a more serious investigation, so we post-poned it after we stabilized the integration of CV in Unified Target)

[Whynotrobot]

Thanks for your detailed response!
I’ll keep using the clipping region for now and will check the 22.12 Localserver release once it’s available.