Data scraping activity doesn't work properly

Help Studio
, , ,
1

Hi,

I’m working on a project where I have to use data scarping activity from website using google chrome, and then store the result in an Excel file. The process is straightforward, and when I run the workflow, sometime it extract the data as expected and return the result in the Excel. and when I re-run the workflow again it’s not extracting any data and returns empty result without any changes in the workflow.

Note: No matter how many I re-run the project, it can be works fine in any attempts.

and sometimes it’s not working at all.

See the workflow below:

Capture
Capture526×654 29.9 KB

I have used Delay activity as a solution but still there’s no guarantee that it will work properly from the first attempt.

Appreciate any help.

2

hi @Shahad

Could u please share your .xaml for more helpful.

Thanks,
Harivishnu

3

This can be caused by non-reliable selector of the targeted table.
Sometimes it is working, sometimes it does not. If you are scraping a public website, this can be even a built-in protection against scraping.
Search for “Selector” in properties of Extract Structured Data. Open that selector in UiExplorer, try to refresh the page and see if it is still pointing at the correct target.

you can play around with this property in case we have the feeling that the table gets scraped sooner than it loads. this may help:
image

You can share your workflow and/or the selector of that activity with us, that will help.

4

Hello @Shahad

Can you please confirm data is always there in the table to scrap?

Also please provide enough delay before and after the Data scrapping. Also, if you are extracting from multiple pages, please provide a delay between pages a higher values if there is any loading issue.

I think here delay is causing the issue and it can be resolved by providng proper delay.

Thanks

6

Hi @anon88744563

Here is my xaml file that contains the data scrape function:

Match Bank Data.xaml (20.2 KB)

7

Hi @Rahul_Unnikrishnan

Yes, the data is always there & I did add enough delay before and after the scraping but the issue still the same.

Let’s say out of 10 run, 3 or 4 stopped after scraping with empty data table as a result. I can’t highlight where is the problem exactly !

I want to ensure whenever I run the project, it should work as expected from the first try, since it is a PoC demo, I don’t want the try to stop in front of them.

8

@Shahad can you move the Extract datatable outside of excel scope and add it directly under the Application scope.

After extraction you can use Excel scope and write range

Thanks