Parallel http request from datatable

Help AI Center
, ,
1

Hello everybody.
I have a question for you.
I have a csv file containing >10.000 rows.
My goal is to use an api for each row of this csv.

If i use the standard logic to read csv and create datatable and after use for each row and inside use http request activity the time consuming of this project is very very long.

It’s not possible to use a sort of parallel activity for http request.
Like 4 or 5 http request in the same time to deacrese time?
Do you have some idea??

thanks a lot

2

Hi,

If it’s unnecessary to use UiAutoamtion in your process, we can run this as Background process.
It might work for you because multiple background processes can be launched at same time.

Regards,

3

You’re processing 10,000+ rows and performing steps for each row. That’s going to take time. You can’t expect it not to. The way you process faster is running multiple Jobs, all pulling from a Queue. So your dispatcher should read the CSV file and create items in a queue. Then your performers pull from the queue and do the HTTP and other steps.

4

Hi ,

I have a similar issue … So, you mean one robot can run multiple jobs at the same time in the same queue !? how robot will handle senario like that !! If I assume that I have one queue with more than 20,000 items then multiple jobs can done with one robot !? how items will be separated in these jobs !?

Thanks in advance…

5

Well yeah, it’s the point to queues. The Get Transaction activity and Orchestrator are designed so that two jobs cannot get the same queue item at the same time.

6

so one robot with a single license can simultaneously run multiple bot ?

7

No. I didn’t say that. I said multiple jobs can pull from the same queue and they won’t interfere with each other, because only one job can get a transaction.

8

Ok. Understand.
But the big limit with this solution is to have multiple license.
so, it becomes a no-go soution.

At this point it should be UiPath itself that inserts a new function to allow a system that launches several actions at the same time that have the same source

9

You think it’s UiPath’s job to give you free software so you don’t have to buy their licenses? LOL

10

I simply said that a parallelforeach function (I would say something rather elementary from a IT point of view) is missing.
For a lack thereof to suggest creating multiple robots (with multiple expensive licences) is not the correct solution

P.S. I already currently have 3 unattend bot licences that I pay for regularly

11

Hey @l.sambinelli

Did you find any solution for this scenario? Because I am facing the same situation here :grinning:

Maybe a possible solution is using asynchronous API with for each parallel, but the asynchronous endpoint must be provided.