Selecting PDF Elements

Hello,

I am new to UiPath and have been watching all the tutorial and playing around. I have the latest community version 2016.2.6274

I am having issues trying to select elements in PDF documents. These documents are native and do not need OCR. I can read the full document text, but I cannot use the screen scrapping method to select individual elements (it only select the full document). I have a feeling it may be something to do with the accessibility settings in adobe reader DC, I have tried going through ‘edit → accessibility → setup assistant’ but this didn’t solve anything. I couldn’t find any documentation on any settings required in adobe, was hoping someone might have a suggestion?

I tried exactly the same thing on an old laptop and it worked (using the same pdf), I compared settings and could not see anything different, apart from ‘accessibility → change reading options’ is disabled on the new laptop (not sure if this could be an issue). Current computer is a surface pro, could the touch screen cause issues (although I can’t see why).

I have tried running both UiPath and Adobe reader as admin, still no luck.

Thanks for any help

3 Likes

In UIExplorer (top toolbar), try changing the framework from the little gear icon, and try indicating and see if that works.

Thanks for the quick reply :smile:

I had a play with that UIExplorer, one of the frameworks selects the whole document, the other looks a bit better in that is has smaller selection boxes, but they don’t seem to match the elements on the screen. It’s like the selection is offset from the correct position.

Could a high resolution screen or adobe settings be causing any issues?

Selecting elements on other apps seems to work fine, its just Adobe that has the issues.

I have been digging away at this issue some more without luck.

I tried it out on a surface pro 3 and it has the same issue, cannot select elements on a PDF through Adobe Reader. I have tried it on another laptop and it worked fine (second machine I have got to work). There must be something different on a Surface Pro that is causing the issue.

The one thing I noticed on the laptop the first time you try selecting an element Adobe pops up a message “Accessibility Setup Assistant”… Adobe Acrobat Reader DC has detected your first user of assistive technology with… This pop up does not happen on the Surface Pro, but I can select it manually but it does not fix anything.

It seems this setting in Adobe ‘edit → accessibility → change reading options’ may be an issue? I found a description of this as “You use this command if you have a third-party screen reader program installed and wish to adjust its settings. If you don’t have one, this option remains grayed out”.

Is there some screen reader that UiPath must installer to get this working? but for some reason failing on the Surface Pro’s?

I did a test on the working laptop, uninstalled Adobe Reader and UiPath. Install Adobe Reader and the ‘change reading option’ is grayed out. Install UiPath, install PDF package, select element and that ‘change reading option’ is enabled… this is what makes me think UiPath is not installing something correctly with Adobe Reader.

Any help is much appreciated, I have had no luck working out a solution.

Cheers

1 Like

Hi @craig,

Can you check as per below settings and let me know.

  1. Open Adobe Reader DC – without a document/s being open – and navigate to Edit > Accessibility > Setup Assistant

  1. Check “Set all accessibility options” and click Next

  1. Compare your settings against the below screenshot

  1. Compare your settings against the below screenshot

  1. Compare your settings against the below screenshot & record any differences

  1. Compare your settings against the below screenshot & record any differences

Regards,
V

7 Likes

Hi @Vikas.Jain,

All setting are the same except ‘Confirm before tagging documents’. I unchecked this but it made no difference.

cheers

That’s strange.

I hope you have restarted Studio as well as PDF reader?

Regards.
V

yep, even restarted the machine! :slight_smile:

Hi @Vikas.Jain,

We have done another test, someone in our office just bought a new machine (Dell XPS 15), so a laptop with a touchscreen. The PDF data extraction is not working correctly, so it will only select the whole document, can’t select individual elements.

The interesting part is this laptop has a touch screen, so we can’t seem to get this working on any PC with a touch screen.

This is quite important to us, we are consultants evaluating UiPath and looking to use the product for our clients (who would purchase full edition). We can’t currently demo PDF functionality.

Any help would be appreciated.

Thanks

@adrian Can you advise here?

Regards,
V

1 Like

Sometimes Active Accessibility fails on PDFs for unknown reasons. Here is a related topic that could fix the issue:

Thanks @Vikas.Jain, @adrian for the quick replies.

I have already seen that post and already tried to update the drivers, but there was no new versions so nothing to update.

It’s interesting that the GDI driver has caused some issues, and we have noticed the issue only on touch screens.

Cheers

1 Like

Hi Craig,

Great timing, I’ve just been updating my previous thread with this exact same issue & noticed your post.

We’ve had the same problem with a client’s system for weeks, but discovered only this week that their laptop is actually a dual laptop/tablet, which started us down the touch screen path.

I’ve been able to replicate the problem on another dual laptop/tablet device & unfortunately, disabling the touch screen functionality (via Device Manager) doesn’t help.

Also, we’ve confirmed today that this is not limited to PFD documents… it also impacts UiPath’s ability to identify elements on standard desktop applications.

Hopefully, there’s some other system setting that can be modified, let see… @Vikas.Jain @Adrian

Cheers,
Dan

Hi,

For most applications we use Active Accessibility to identify UI objects on the screen. This is also used for PDFs. Sometimes AA stops working on PDFs but that should not affect other applications. If this is the case you can try the following command from an elevated command line cmd.exe

regsvr32.exe oleacc.dll

It is possible that only the on-screen selection fails to properly highlight UI objects (which can happen on multiple displays with large DPIs) but identifying objects using selector could still work. To test this you can use UiExplorer and try to expand the UI tree to see if you can find UI objects in the hierarchy.

What kind of apps did you test?

Regards,
Adrian.

3 Likes

Hi @adrian, @dlp1980,

I have tried a lot of things in the past week, disabling the touch screen didn’t work for me either, nor did using UiExplorer (i have only been trying this on PDF’s).

I will try the ‘regsvr32.exe oleacc.dll’ command when i get back to the office on Friday, i don’t have my Surface Pro with me to test.

Cheers

Hey guys,

Ok, so I’ve made some interesting progress with UiExplorer, as follows…

From the top menu, click UiFrameworks (last button on the right) and change the selection from “Default” to “UI Automation”… after making this change, I can select individual elements as expected! This works for both native PDF’s, desktop applications, etc.

The problems?

1/ It only works whilst operating out of UiExplorer, with that setting changed as per above… when I go back to the regular Studio interface & try to use a recorder and/or individual activities, it reverts to treating the target UI like a Citrix environment.

2/ Copying the validated selector code from UiExplorer & then pasting it into the Selector field for an activity (ie. Get Full Text) results in COM Exception.

Any thoughts?

FYI, I tried the regsvr32.exe command from an elevated command line & it didn’t appear to have any impact (assuming that I did it correctly).

Also, the apps I’ve been testing are just basic Windows desktop apps - Win Explorer, Control Panel, Excel, Acrobat, etc.

@Adrian @craig

1 Like

You may try inspect.exe tool from Microsoft to see if AA and UIA work on your machine

https://msdn.microsoft.com/en-us/library/windows/desktop/dd318521(v=vs.85).aspx

The tool comes with Windows SDK but I uploaded it on my dropbox so you don’t have to install the SDK:

Also you could use Highlight activity rather than Get Full Text to see if the selectors are actually found.

Adrian.

4 Likes

Thanks for the inspect.exe tool, I’ll give that a try & report back.

I used the Highlight activity & it did identify the selector correctly… also used Get Text, instead of Get Full Text & it returned the value correctly!

At least there’s some workarounds… I haven’t needed to use UiExplorer that much up until now, but I’d assume it’s not dramatically slower than working directly out of Studio, would that be right?

Dan.

1 Like

Hi @adrian,

The inspect.exe tool seems to work on my surface. If I hover over an element on the PDF it will pull out that text correctly.

None of the suggested methods above have helped within UiPath (ie regsvr32.exe oleacc.dll). Also @dlp1980 instructions did not work for me :frowning:

I have noticed patchy results for web scraping as well, sometimes it works, other times it does not.

@dlp hopefully you’ll need UiExplorer only in limited scenarios where the defaults generated by Studio are not enough (or when the selection is not working properly like in your case)It will be a bit slower from a usability point of view; you need to work more to get the same results that usually the recorder produce.

Adrian.