I have a workflow which performs data scraping from a website in chrome browser. It navigates to more than 10 pages in the website to do the data scraping. I run this workflow 5 times a day. The first time the program is run, the bot executes it in a nominal time. But by the time the execution reaches 5th time, the bot takes more than 10 minutes to run the same program. What could be the problem?
Buddy there are two possibilities
- Usually when we work data scrapping from a webpage …it usually depends on the time taken by the page to get loaded…and its all based on the network connection we have…
- Then if we have any image automation along the process you have or any delay activity, it would lead to additional time taken by the bot…
The reason is image automation usually takes more time than other data fetching activities…but still if no image automation activity is involved in the workflow, then check whether there is any delay activity been added and when we have delay in process line…with some time given as input…the process will wait until the delay time is COMPLETELY consumed…but if we use on element appear, or on image appear kind of activity…and if timeoutMS property of each activity is mentioned with some time…the activity wont wait until the full time to be utilised rather will wait till the element appears and continue immediately once element appear…so the time gets saved buddy…
Still if you dont have any delay or image activities but still taking time…dont worry buddy you are going good…its all because of the network…and system speed to recognise each element buddy…
Normally clearing cache (and maybe even history and other form data) helps in the speed, and as Chrome is not the default recommended browser for UiAutomation it might take some time.
Also, try changing the property to ‘Interactive’ for all the other activities except for an initial page load check and this might save you some time.
I am sure you would have check the Delays between pages etc.,
My workflow is based on excel automation, web automation is not involved and the time difference for the first run and the 3rd or 4th run is so much.i.e if it takes one hour to run the for the first time and it is taking 3-4 hours for the 3rd run, and the input data is same.what could be the reason for so much time difference?
Hi there @vinitha, welcome to uipath community…!!
Can you share your excel input file and .xamls if it’s not confidential.?
I see that you said the input data is the same, so is it the same workflow with the same input data that takes increasingly more time?
PS : From now on ask an independent question so that more Uipath developers can also see your question in the forum and address your issues, as replies will be rarely seen by everyone
but why does the same workflow take less time in the first run but takes more than 10 minutes for 3rd run with the same network speed? Everyday…first execution is the fastest…all other executions take a long time to complete. Sometimes I have observed that after data extraction from web is complete, the remaining activities take more time to execute.
By clearing cache, do you mean deleting temp files? I tried doing that but more temp files keep generating each time I run the program. How can I make that stop? And why does the workflow execute quickly the first time but slows down after 3 or 4 times without making any changes to the program?
this would help you…
we had this issue very long back…
Sometimes the issues are or might be due to excessive logging, or writeline or too many requests to the explorer in loops. You can run a batch file to clear all temp files whenever the bot runs.
Try this and let me know, because whenever I encounter this issue these are the steps that I take.