Click and Type Into are going to the wrong fields intermittently

[Ashley_Gates_US_ADVS]

Hello,

I have a bot that is updating fields within a pop-up wizard in a page, as illustrated below. For some reason, the bot intermittently puts the wrong values in the wrong fields.

For example, it will usually populate AccountingPeriod and Ledger correctly, but then it will clobber Ledger with Company, then clobber Ledger again with LoB, but Account might go in correctly, etc. The scenarios are not always identical, but usually they tend to be bunched up near the top (i.e. clobbering fields near the top and missing fields near the bottom).

The fields are being highlighted correctly in red, both in UI Explorer and in the Selector Editor. (I also added some Log Messages to record both the Selector details for each field and the value it was trying to write to that field, and the Selector logic seems fine.)
Moreover, there are no error messages when the program executes.

The Selectors used for each of these fields re nearly identical, with only the rowName parameter being different. For example, here are the Selectors used for the first six fields:

Taking the first field as an example (i.e. AccountingPeriod), here is a screenshot of the Selector Editor:

And, here is a screenshot from UI Explorer for the AccountingPeriod field (again, all of the other fields are very similar):

Again, I think the Selector logic is solid. It seems to me like this particular wizard is causing the Click and Type Into activities to misbehave. As a work-around, I’m contemplating modifying my program to tab through the fields rather than using Click and Type Into, but that approach has some challenges as well (i.e. I would have to inspect each field to know what it is, since the fields may change).

If anyone has experienced a similar issue, where Click and Type Into are executed into the wrong fields, despite the Selector logic being correct, I would be very interested to know how you solved or worked around the problem.

[msan (Michel SAN)]

Hi,

Your selectors look nice. I don’t have any idea at first glance for troubleshooting this issue but I wonder if your TD elements have any INPUT child that might be a better target.

You could also put an highlight activity before the Click and another before the TypeInto.

I presume that you’re using a Click activity because the option ClickBeforeTyping didn’t produce expected results and that you carefully tuned TypeInto’s options.

[Ashley_Gates_US_ADVS]

Hi @msan,

Thank-you for sharing your ideas. First of all, I have some new information to share. After carefully observing it many times, the behavior is not random. It populates the first two fields correctly (i.e. AccountingPeriod followed by Ledger), and then the next 6 fields all clobber the Ledger field, and then the remaining fields 7 through 11 are all shifted up to the empty fields 3 through 7). Below is a screenshot to illustrate. (Note I interrupted it before Currency Type). By the way, I did try zooming in to increase the size of the screen, but no luck. (I’m tempted to try populating the fields in a different order in case spacing is an issue, e.g. 1, 6, 2, 7, …).

Thanks for your idea about adding the highlights. I had previously added a hover before click, and the two seem to be conveying the same information (i.e. it essentially hovers right where it is about to click and type into). I did notice that the highlight was off by a few millimeters relative to the field; I don’t know if that is significant or not.

Thank-you for your suggestion to check whether the TD elements have any INPUT child that might be a better target. I will try to research that.

Regarding my ClickBeforeTyping, believe it or not I have to use BOTH a Click activity AND a TypeInto with ClickBeforeTyping selected. Through trial and error, I found that to be the only way I could type text into those fields. (I also tried other things like adding delays to make it super slow, hovering over the field before clicking on it, etc.) No matter what I tried, I couldn’t seem to get around the need for a Click followed by a TypeInto using ClickBeforeTyping. (And I also had to follow that with clicking outside grid before proceeding to the next field.) Normally I don’t have to go through such elaborate lengths to put data into fields; this is a particularly stubborn page.

[Ashley_Gates_US_ADVS]

Sorry, I forgot to include screenshots of the Click and Type Into properties. Here is the Click. (Note: I currently have it set to Double Click, but single produced the same result.)

And here is the subsequent Type Into:

[msan (Michel SAN)]

You might face a validation triggered at inputs’ values change (the actual trigger(s) can vary). You can also encounter hidden fields (raw data, html formated data, etc).

In this situation, I would pay attention to trigger that update event.

[Ashley_Gates_US_ADVS]

The fields do have validations. When I populate the page manually with the same values (but in the correct places), there are no issues; I am able to click the “Next” button and proceed without any issues. When the bot runs and puts the fields in the wrong places, it still proceeds and the error messages don’t appear until the page is complete and the “Next” button is clicked.

[msan (Michel SAN)]

Yes, I was facing similar issue in my first ever experience with UiPath: the selectors were correct (then no exception raised) but the focus was not moved and all typing was in the same field

Since you have to disable Simulate Click/Typing and you’re in a web page, you might consider JavaScript “injection” but that require some “reversing”.

[Ashley_Gates_US_ADVS]

I’m sorry that that was your first ever experience with UIPath. It sounds like it was not a fun introduction!

What would the JavaScript do exactly? Also, can you elaborate on “reversing”?

[msan (Michel SAN)]

You would update the values by javascript and force the triggers if needed. It is not really reversing but you’ll have to debug some javascript

[Ashley_Gates_US_ADVS]

Thanks for that suggestion!

Hey good news, I think it is working now (except for the last Currency Type field, but hopefully that is a one-off issue that I can figure out). When I changed the Click activity to SendWindowsMessages, it worked beautifully!

Thanks for all your help, I appreciate it.

[Ashley_Gates_US_ADVS]

Sorry, I may have marked this as “complete” prematurely. It worked correctly one time, but then afterwards it reverted to its original behavior.

[msan (Michel SAN)]

Did you change your TypeInto activities’ options too?

[Ashley_Gates_US_ADVS]

No, those I had to leave as-is. Otherwise, it wouldn’t type anything.

[msan (Michel SAN)]

Could you type someting into a field and select it with the UI Explorer and share the Visual Tree, please?

[Ashley_Gates_US_ADVS]

[msan (Michel SAN)]

Can you please dig into the Company’s TD to see all its descendants?

EDIT: It might also be interesting to do that, interact with the ui then come back to the UI Explorer and see if any change was applied.

[Ashley_Gates_US_ADVS]

I believe the Company has only one descendant:

Here is the last line which was cut off:

[Ashley_Gates_US_ADVS]

Regarding your edit, interestingly when I highlight the child element, the red rectangle covers only the input value within the field, rather than covering the entire field.

When I click on the field, notice that on the left is the actual value that is input, and on the right is a small square icon (which can be used for lookups).

[Ashley_Gates_US_ADVS]

The following selector logic seems to work as far as highlighting only the text within the cell rather than the entire cell:

I will try rewriting my code to target those children instead and let you know how it goes.

Thanks!

[Ashley_Gates_US_ADVS]

Ok, it was a good idea, but sadly it is behaving the same as before.