How to read complete data from web in table, when data is 50K?

Help
, ,
1

I’m scraping some data from the web using DATA Scraping, but pursuing a problem when my data have the small amount in table, so that time data scraping method is working with next button. But when my data is having 50k or more then it so it isn’t working. The robot starts and scraps the first page but when it goes onto next page it should stop and move on the next data table.
so have the solution for it at a time waiting for when the next button is hidden.
Please help me to check it out.
Thanks in Advance

2

Hi @bagishojha,

Have you solved the problem ?

Regards
Balamurugan.S

3

I haven’t solve.
if you have some idea about.
please share with me
Thank you!
Regards
Bagish ojha

4

Hi @bagishojha,

Can you share the thing that how you have solved it.

Regards
Balamurugan.S

5

HI @bagishojha,

After clicking Next button, check the selector for the table is having the same using UiExplorer .

get the both the selector before and after clicking next button and compare the selector.

Make it dynamic selector.

https://studio.uipath.com/v2017.1/docs/selectors-with-wildcards

Regards,
Arivu :slight_smile:

6

i have made my selector dynamic and u know what the problem when the data is too much(50k or 80k) then its takes time to go one page To NEXT page so the data scraping activity time is out and robot leave this extraction and go back to the NEXT transaction.
So, Have you some idea about wait element like that.
Thanks in advance.
Regards
Bagish ojha

7

No, I can’t.
but u know what the problem is when it’s trying to load next page then, in loading page that web page takes a long time to open that’s why it happened. so have you any solution to the robot can wait until target, not shown.

8

Hi Bagishojha,

If you think you cannot predict wait time and if it takes longer, try using other wait activities like

1). ‘Wait Screen Text’ - Waits for a specified amount of time for a string to appear in the terminal screen.
2). Image Exists - Checks if an image is found within the specified UI element.

and then continue scrapping data. You have to make sure, the text/image you are searching should occur constantly in all pages.

Thanks,
Suresh.K