Hi all,
i want to scrap reviews from zomato site. the page is dynamic means in one process we have different restaurant in next process the restaurant is different. I have made the selector dynamic and able to validate. Scrapping only 50 reviews and waitforready parameter is “complete” state.
example of review url :
https://www.zomato.com/hyderabad/shah-ghouse-cafe-restaurant-tolichowki/reviews
Still sometimes i m able to scrap data while sometimes it don’t.
i have used input dialog to give runtime url of restaurant review page so that i can check for random diff restaurant .
attaching the xaml file.
mainfile.xaml (55.9 KB)
project.json (1.5 KB)
please let me know whats the issue. i checked previous such issues but didn’t got the resolution
SowmyaLeo
(Sowmya Sivakumar)
January 16, 2020, 9:25am
3
@Abhinavpandey
Please can you add a screenshot of the data scraping.
I can see an inuput dialog where the url will be entered. The next activities are missing.
1 Like
sure @SowmyaLeo
Part 0 : ENTER URL IN INPUT DIALOG THEN navigate activity to go to that url
Part 1 : click on all review button then message box for confirmation then attach window to scrapping part
Part 2 : do data scrapping and then message box for confoirmation
Part 3 : both review container and next page button to extract review of next page is valid
You could also use a find children activity to scrape the reviewsScrapeReviews.zip (2.8 KB)
i tried to run but it is giving error and also for few it scrapped data. it is extracting whole this like this
apart of user review , comment floow share all these things are also coming.
one question what are we doing over here if you can explain this
The try catch should handle that error. It should simply continue to the next review.
See here, for me, it reaches the end and no error
For the Assign, it is grabbing the idx number of the selector, if it is null, the selector is not what we need, so it is skipped to the next one.
As for the full output, you should be able to do some parsing of the string to get whatever elements you need. I can add an example later
like comment share all opetions are coming . its scrapping whole data on page. i want want reviews
A bit rough and ready but gives you the following CSV of reviews. You may adapt it to grab other elements such as reviewers name etc, simply by using the Regex Matches activity and grabbing different elements of the string
ScrapeReviews.zip (3.5 KB)
You should have what you need to get the various elements and parse the resulting information in a way you can report with.
1 Like
@Abhinavpandey
Hello,
I make a proposal for something similar.
Please check this post.
@reshmaprabh
It would be great to share the solution here with screenshots or just explained it.
If you see my post, when I found my own solution - I post the solution.
Because tomorrow maybe someone will be with task like this and you will help him.
But this is only a proposal to you.
I believe this is good practice for all people here.
solution is again by find children where whole data is coming with follow share comment like these kinds of data,
i just need to scrap the reviews
But in the example I provided, those items of data have been removed. You only have the reviews:
1 Like
Thanks @ronanpeter
I tried it
It’s giving error
Tomorrow I will try to debug it and also try on ie browser the same
Thanks for your help