Currently i am working on a process which requires me to pull data from Database from stating of the year till today, based on month. i have the query and i have tested it. Volume of data is around 700k rows for one month. My query is taking around 45mins to get that much data. As i want to pull the data till current month i have used a parallel activity to run multiple queries at once. As the first query is taking time, it try’s to execute second query and so on so forth. this is taking lot of time to fetch information.
my questions is:
Is there better approach for this other than using multiple robots to fetch data for different months?
As currently i am using one connect database and using that connection on multiple run queries in parallel. Will there be any difference in time if i use different connect database activities for different run query ?
Your ideas and comments are much appreciated. Thank you
Adding to the above information. At the end i need to paste the information in excel file. so i fetch it for two or more months or for a year, data cant be inserted into excel sheet as the excel support limited rows in single sheet. I have used logic to keep each month data in different excel. kindly consider the volume of data and excel when presenting the ideas and solutions.
In the SQL query get only rows for one month, write to Excel, get next month, etc.
Either way, 770k rows is a lot and it will take time to read and write all that.
If your goal is simply to get the rows from a database and write them to Excel, then I would suggest using UI automation to navigate around in the database client app, run the query, and export to Excel from there.
Hello @postwick Thank you for your reply and i respect your idea. True, using navigate we can move forward in DB and then extract information needed. My current goal is to fetch information from the Database only. The current query is designed to get information from From date to To date. So i am passing the start of month date and end of month date as inputs. So its similar to what you mentioned. Thank you for your reply.
If speed is the issue then it is better to user scheduler on the db to send the data periodically or daily to export it …this way it can be faster
Alternately if you need from UiPath only then instead of parallel activity try with a loop with month on month data …as if you use paralle activity…there might be miltiple request placed to db at the same time whichight end up with more resources and load on db
ashokkarale
Anil_G
lrtetala
B_o_D
postwick
shaik.muktharvalli1
marina.dutta
Evelyn_Lu
Ejaz_Ab
ppr