Datatable wont iterate through links. Can't add column data. Please Help

Hi so Im trying to use a inintial data table with a list of URLs get the bot to scrape a table off each of these URLs. I then want to add a data column with the URL that the data is related to but my worflow only scrapes one game and column cant add as there is an assign error for undefined item. Please help. Workflow and excel sheets attached here UI 6 Rounds Input.xlsx (9.1 KB) Player Stats v1.1.xaml (16.4 KB)

I’m taking a look at your Workflow now. If i understand correctly you are looking to…

  • Navigate to the Link listed in each row of UI 6 Input

  • Scrape the Table found in each link

  • Build a new table with all the Scraped information and add the Link?


Update …

Try something like the attached. Its not optimized but it should work depending on how you set up your screen scrape. It navigates to the Link listed in each row of UI 6 Input. Scrapes a table on the page, saving that to a DataTable. Writes that DataTable to an Excel Sheet named the first column of the UI 6 Input, for each row in UI 6 Input. Player Stats v1.1.xaml (10.8 KB)

Thanks for the reply. Yes this is exactly what I’m trying to achieve. I’ve tried your method and it loads all the web pages. I made a few tweaks I added an Open browser above the navigate to section sing the link variable as the URL which seems to have worked but I have a slight problem with the filepath on the write range where it says “row.Item(0).ToString”
.
Also I’ve got problem with the actual tables where you need a click to get both the home and away table but Im not sure if I can combine these when I extract them or if I need to extact them later. I have tried using the data spanning multiple pages but with no luck so i dont know if i have to extract both seperately and combine them later. https://www.foxsports.com.au/nrl/nrl-premiership/match-centre/NRL20080101/stats.

This is what the workflow looks like now Player Stats v1.2.xaml (12.2 KB)

Is it safe to assume you are looking to DataScrape the Player Match Stats?


Update…

To get your scrape to work you will need to use a dynamic selector for the NextLinkSelector in the ExtractData Activity Properties panel. I found this selector to work…

<webctrl aaname='TeamName' tag='BUTTON' parentclass='fiso-mcfootball-match-player-stats-button-row' />

You will need to set the aaname to a dynamic variable, see this post if unaware of how to do so (How to use variables in selectors in Studio (dynamic selectors) without string manipulation!).

TeamName should be set to the Away Team as the home team load by default every time.


From here you will have a DataTable of the Stats from the Match played, called DT_MatchStats. Take that DataTable and use a For Each Row activity to loop through each each row in DT_MatchStats and add each row to your DT_Final.


Outside of the Original For Each Row activity, use a Write Range Activity to create your final XLSX.

image


image

Thanks I’m ok on the dynamic values I think as I was using them within the browser field earlier as part of my inintial solution. One last question… what are the inputs under the “Add Data Row” section? Thanks for the help this is all super useful

For the Properties of the Add Data Row activity, set the DataTable to DT_Final and the DataRow call the Items from DT_GameStates you would like to save (examples below)

image
image


Update…

If DataRow starts giving you an issue, use a Build Data Table activity outside of the Navigate to Webpage sequence and set your columns accordingly .