Data Scrapping not working properly

Hi all,

i want to scrap reviews from zomato site. the page is dynamic means in one process we have different restaurant in next process the restaurant is different. I have made the selector dynamic and able to validate. Scrapping only 50 reviews and waitforready parameter is “complete” state.
example of review url :
https://www.zomato.com/hyderabad/shah-ghouse-cafe-restaurant-tolichowki/reviews

https://www.zomato.com/hyderabad/mudbrick-kitchen-and-bar-jubilee-hills/reviews
Still sometimes i m able to scrap data while sometimes it don’t.

i have used input dialog to give runtime url of restaurant review page so that i can check for random diff restaurant .

attaching the xaml file.
mainfile.xaml (55.9 KB)

project.json (1.5 KB)

please let me know whats the issue. i checked previous such issues but didn’t got the resolution

anyone can help on this?

@Abhinavpandey

Please can you add a screenshot of the data scraping.

I can see an inuput dialog where the url will be entered. The next activities are missing.

1 Like

sure @SowmyaLeo

Part 0 : ENTER URL IN INPUT DIALOG THEN navigate activity to go to that url
pART0

Part 1 : click on all review button then message box for confirmation then attach window to scrapping part

Part 2 : do data scrapping and then message box for confoirmation

Part 3 : both review container and next page button to extract review of next page is valid


You could also use a find children activity to scrape the reviewsScrapeReviews.zip (2.8 KB)

i tried to run but it is giving error and also for few it scrapped data. it is extracting whole this like this

apart of user review , comment floow share all these things are also coming.

one question what are we doing over here if you can explain this
int

The try catch should handle that error. It should simply continue to the next review.

See here, for me, it reaches the end and no error

For the Assign, it is grabbing the idx number of the selector, if it is null, the selector is not what we need, so it is skipped to the next one.

As for the full output, you should be able to do some parsing of the string to get whatever elements you need. I can add an example later

like comment share all opetions are coming . its scrapping whole data on page. i want want reviews

A bit rough and ready but gives you the following CSV of reviews. You may adapt it to grab other elements such as reviewers name etc, simply by using the Regex Matches activity and grabbing different elements of the string

ScrapeReviews.zip (3.5 KB)

You should have what you need to get the various elements and parse the resulting information in a way you can report with.

1 Like

@Abhinavpandey
Hello,
I make a proposal for something similar.
Please check this post.

solution is again by find children where whole data is coming with follow share comment like these kinds of data,
i just need to scrap the reviews

But in the example I provided, those items of data have been removed. You only have the reviews:

1 Like

Thanks @ronanpeter

I tried it
It’s giving error
Tomorrow I will try to debug it and also try on ie browser the same
Thanks for your help :slight_smile: