Check if image or video exists on screen

Help Studio
, ,
1

Setup:
I am connecting to an app on my android phone using Any Desk.
UiPath sees the image of my phone’s screen.

With CV GetText, I am able to read all content on the screen. The content can be an image or a video or plain text. Output of CV GetText is all text, plus some random characters instead of the image or video (some thing like: " W2W97”T“ 3W§m§q ”37931209 "£ 3/391???“ %//y mar» gamuvé’w Mama/*3; v. “‘3’ ’ .,~’ … , -; :91 25/5, ,"\\-”7 ’94:; 94, 433% A 42:2"? 55/ 4-3: I ‘z‘fifflflé’gm 7/, ,7 7v 3;.").
Again, the images or video are random and the text grabbed (actual text or image text from CV GetText), is random.

I am looking for a way to figure out if any image or video exists on the screen? I was not able to figure out a pattern or any delimiter I could use to break up the text.

@Palaniyappan @bcorrea @loginerror - any suggestions?

2

Hello there,

I’ve recreated the same situation and I have noticed that CV GetText works properly. The issue occurs when you use “Get OCR Text”. You might want to check if you are using “CV GetText” or “Get OCR Text”.

Hope this helps,
Cheers!

3

@Enache_Andrei - I played around with CV GetText as well. What happens is that it just neglects the image or video (which is expected). So, from the output of CV GetText, I know there is some text, but there is no indication of if there is any image or video.

Appreciate the effort you put in to test this.

4

Solved it by looping through the screen vertically and getting and saving pixel color after every 50 pixels (vertically).
If any of the saved pixel colors were not equal to text or screen background color (white, gray, black), assume it is an image or a video.
Not a 100% accurate method, but does the job for me.

Thank you!

2 Likes
5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.