Extract Structured Data is slow

Help
1

I’m extracting stock values for given company from Yahoo finance web page. It works, but it takes around 34 to 35 seconds per each web page. For 42 pages it becomes about 25 minutes. I identified that the bottleneck is UiPath.Core.Activities.ExtractData but not page navigation. Is there a way to increase the speed? I tried using Build Data Table activity before ExtractData activity, to prepare data table columns, but no gain in speed. I’m using community version 2018.1.

2

If web page has “Next Page” button, then Extract Data Activity will click that button till the last page. Since there is no “Next Page” button on the last page, the activity will throw an error (which seems wrong to me). If ContinueOnError = True, then that error will not stop the workflow. Before that error is thrown, there is a waiting time, which is controlled by TimeoutMS, which defaults to 30 seconds. Decreasing that timeout will increase overall speed; however, too small amount of delay may result to the activity not being able to extract all data.

3

Hi there I’ve faced the same situation where data extraction with next page is becoming slow. Have you found a solution? I tried delaybetweenpages reduction but not sure if page delay to load will cause additional error…

4

Hi @amnuay and @synergetic,

Can you try the WaitForReady property as Interactive or None.

image

Regards
Balamurugan

2 Likes
5

@balupad14 Thank you it did solve! However what exactly WaitForReady parameter behave? The whole page loaded? or any specific document you have read?

Thank you very much by the way.

6

Hi @amnuay,

Take a look the below.

Regards
Balamurugan

1 Like
7

Thank you very much. This help us a lot on managing activities behavior. I have another maybe feature request for UiPath team will post soon then.

8

Hello,

I have tried this but still i face delays in extracting data.
where I tried only 3 items (Name, City, URL) from a single page.

Thank you.
Akhil