Save to Excel while RUNNING scraping

Hello, buddies. How do I make Uipath write to Excel or CSV file the data that is CURRENTLY BEING scraped by Data Scraping activity?

I noticed that the scraped data is wrote on the output file only after the END of the process, but I would like to make data scraped be wrote to the file while bot is currently RUNNING

Hi bipenett,

I believe the output of Data Scraping is always stored in a DataTable Variable. You can write it to excel or csv file after the activity.

Data Scraping in UiPath

Regards,
Austin

exctly, it can be write after activity, but I would like to write it DURING the running of the activity. Do you think this is possible?

Hi bipenett, it is not possible if I am not mistaken. Just wondering why do you want to do it during the activity?

There are two activities in action:
1- data scraping
2- writing to excel (output)
They cannot run in parallel in real time. An alternate could be to break down the scraping activity into smaller logical pieces and you can write the results back to excel after end of each logical piece…

I am still wondering the real intent behind the ask… if you would like to share…

1 Like

Hi @bipenett

As per for ur requirements u can try the following

After extract datatable , u can use append range to append the datatable to the excel file

It would be one approach u can try

Regards

Nived N :robot:

Happy Automation :relaxed::relaxed::relaxed:

because i wish to scrap a website containing infinit pages, which loops to the the first page (which has dinamic content) once it reachs the last one

hello buddy, thanks for replying. How can I perform the mentioned “break down” during the scraping?

i need to scrap a website containing infinit pages, which loops to the the first page (which has dinamic content) once it reachs the last one

Are you getting count of Total records anywhere in your webpage?

Do you have count for total number of pages?

no, actually im only scraping the content and clicking “next button” (which loops infinitaly)

@bipenett - logical break down can be done in many ways… a few considerations could be -

1- keeping a running count of number of rows \ no of items scraped and break after every 10 records \items.
2- you can also consider “>next page” event as your logical break down point.
3- you can go based on object memory size as well but that get too complicated.

One of the first 2 are reasonable break down conditions, you can try. I hope it’s helpful.

1 Like

thanks buddy, i think it helps!

glad to know… it’s helpful for your use case.