Animated Image based questions

Help
1

The scenario is - I am trying to test a simple web based game by sending some hotkeys to perform a specific action that needs to be performed by an image which will be moving around and the web page has the following components…

  1. Web page with webctrl
  2. In the webctrl there is an animated image or an image which moves upon key pressed.
  3. This image will not stay in one place it will be moving within the boundaries of webctrl.

How can we

  1. Get the relative images data so when it moves I can highlight that particular image
  2. Also, how not to keep the focus on the background, so the relative image is always in focus and then we can perform action upon this.

What has been tried.

  1. Find image doesn’t seems to work properly. Or rather I should say it will only work if the image position is static
  2. Highlight doesn’t seems to focus after using “Set clipping region”

I am totally lost and request someone to help me out with this or provide some pointers.

2

When selecting the part of the image in “Find image” have you made sure that section of the image never changes, and doesn’t include any background?

1 Like
3

Well, that is the problem! When an image is moving or in action or animated and the background keep changing, how can we select that image? OR do you mean to say select portion of the image and be cautious not the select the background? I have tried this too but seems it doesn’t like animated stuff :slight_smile:

4

Find image and image exist will not work for this scenario, if you have webctrl try with selector of image which is moving, and first check weather selector is same in all moves of image if it’s same then try to automate it.

1 Like
5

Is there any possibility to provide a source sample please?

6

i will check if i have any sample similar to this and will get back to you:)

7

Yes I mean the second option, pick a static part of the image, if it’s mario jumping for example, grab a small part of his chest/face which never changes no matter his movement.

I assuming you’re trying to automate a web game of some variety?

2 Likes
8

On that note - check the terms and conditions of the game - a lot of devs specifically have rules against botting games, especially if it disadvantages others.

9

Tried this, no luck. Consider Mario jumping, the coordinates will be an issues in my understanding, even if I grab a small part of the character or image. Yes trying to automate a web game. Yes, will keep a tab on the T&C of the game which is hosted on web before proceeding further with this. But it will be a lesson to learn using UIPath to automate as well - Any help is really appreciated. I am this close to solve this so keen on looking for some guidance and ideas :slight_smile:

10

If you’re just looking for an image (assuming you find one static piece of it) it should find it anywhere on screen should it not?

11

I see a lot more (business) use-cases for detecting image within animation , than merely ‘botting mario’ … that said, it should work , but to clarify a bit,

  • use a larger area of screen to ‘detect’ or recognize the static image (match the still image / frame)

  • use a smaller area of the animation (mario cheek) frame to use for matching

I can see this working fine if you iterate the recognition attempts recursively/loop through , say , all frames in a gif or with X seconds time delay of some fraction of the fps of the animation , multiplied by Y number of frames

1 Like
12

Yeah, @Alex_Cross makes a good point.
you should be able to find the image then take it’s coordinates using the element variable that was outputted to determine its location. I don’t know if UiPath can do this fast enough for Mario, but good luck!

13

Also, if you are trying to break records, here’s a good explanation on how it’s possible: How is this speedrun possible? Super Mario Bros. World Record Explained - YouTube
:rofl:

2 Likes
14

@ClaytonM this is too much information to process, but a very good learning :slight_smile: thanks for sharing.

15

@Alex_Cross, well it should find it anywhere on the screen, but we also need to remember that if its only pattern based recognition then it would have been easy, but when we use color as a the primary value to highlight as an image ignoring pattern of the image then there is a challenge. Example, in Mario, most of the background is has a very static color like blue, green etc except when he jumps, identifying here will be quite easy. But when the pattern of the image it self is overlapped with multiple patterns blending into background at the start, then the real challenge comes. - I hope my understanding is correct.

16

@Divyashreem Thank you and look forward to your support.