Web Scraping Issue With Web Table

Help Studio
, ,
1

Hi there, I’m new here so please excuse my poor understanding.

I have scraped a list of data which includes a column of url’s providing additional data which I would like to add into additional columns in my extracted data table. I have automated the opening of each url but am struggling to extract the data on the next page. Using the wizard doesn’t seem to work and manually selecting the data and trying various techniques seems to return blank data. I cannot even extract the data when I start a new sequence simply going straight to the url and attempting to scrape the text using get text, get full text etc

This is one of the url’s I am attempting to scrape…
https://my.rouvy.com/virtual-routes/detail/69181

The image shows some of the data I want to extract…

rouvy snip
rouvy snip1289×546 70.1 KB

My selector shows the following when I have done the click to indicate selector option…

<webctrl tag='TABLE' />
<webctrl tableCol='2' tableRow='2' tag='TD' />

I don’t think I need any help with regards the handling of the data post scrape, I just can’t figure out what it needs to actually extract the distance and elevation figures…

Many thanks in advance,

Ash

2

Hi @Ashley_Howell
so u had the difficulty in extracting the underlined data right?

3

Yes that’s correct.

4

Hi @Ashley_Howell
Try this for
Distance

<html app='chrome.exe' title='Virtual route - IRONMAN 70.3 Chattanooga' />
<webctrl parentid='content' tag='TABLE' />
<webctrl tag='TD' css-selector='body&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;table&gt;tbody&gt;tr&gt;td' idx='6' />

AVG Grade

<html app='chrome.exe' title='Virtual route - IRONMAN 70.3 Chattanooga' />
<webctrl parentid='content' tag='TABLE' />
<webctrl tag='TD' css-selector='body&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;table&gt;tbody&gt;tr&gt;td' idx='8' />

Ascended

<html app='chrome.exe' title='Virtual route - IRONMAN 70.3 Chattanooga' />
<webctrl parentid='content' tag='TABLE' />
<webctrl tag='TD' css-selector='body&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;table&gt;tbody&gt;tr&gt;td' idx='10' />

Well this is the selector i created for the url u pasted in the topic,

Regards,
Nived N

5

That looks more like what I am looking for. I will have to make it dynamic so it works for my other url’s, but I believe I can. Would you mind giving some tips on how you acquired those?

6

Yes
@Ashley_Howell
First after indicating the get text activity, use the uiexplorer to edit the selector

from there only check the attributes that i had shown the selectors like as above

1 Like
7

@Ashley_Howell
did it resolves the issue?

8

Yes I have got the full solution working for distance just now with it appending distance into a new column in my excel sheet. :slight_smile:

Thank you for your help, it took a bit of tweaking to get the selector to work dynamically and the page I was working with was slightly different behind the login, but I am there now and can continue scaling up my little project…

1 Like
9

Like did u used the same selector approach i used ?

10

Yes, I had actually watch some youtube videos of a guy called Anders showing exactly how to do it, but for some reason the element only showing row and column in the selector totally threw me off…

rouvy snip
rouvy snip795×67 13.1 KB

That was the dynamic one I ended up using, but then I had to redo it for the web page in the secure area that was slightly different. Not that I had to, but I am running a few sequences that require login and it makes more sense to treat them all the same… :slight_smile:

11

CSS Selector are great attributes to handles these around, as it helps to locate the element at fixed point.

Mark the appropriate response as solution so that we can close the thread

1 Like
12

:+1: Thankyou again !

1 Like
13

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.