Need help with extracting data using Data Scraping

Help
,
1

Hi Friends, Need help with extracting data from table in a website using Data Scraping.

Here is the scenario, I have to process each parent record individually and there are sub records associated to each parent record. So I created a for loop to loop through each parent record and within that I am using Data Scraping activity to pull the sub records in a datatable. Now sub records doesn’t have consistent data for each parent record like for parent record 1 sub records may have data for certain columns and null for others while parent record 2 sub records has data for certain other column which doesn’t match with sub records data of parent record 1. Due to this the datatable which I am creating for sub records doesn’t have consistent structure and due to which I am not able to extract data from that datatable using “ExtractDataTable(0)(2).ToString”. Also below are the issue with datatable being formed.

  1. It just pulls the data and not the headers from web table and due to which I need to access the data using “ExtractDataTable(0)(2).ToString”.
  2. It shows additional rows other then actual data which are not highlighted in attached snapshot.

Attached are the snapshot to support the above scenario.

Parent_Record_1_Sub_Records.jpg1186×130 40.3 KB

Please share your input how to best handle this scenario.

2

DataScraping_Parent_Record_1_Sub_Record.jpg827×327 91.2 KB

3

DataExtract_Parent_Record_1_Sub_Record.jpg717×485 113 KB

4

Parent_Record_2_Sub_Records.jpg1186×158 47.8 KB

5

6

HI chintansalvi,

You can try like this for extracting data form table,

Datatable.rows(i).item(j).ToString

7

Hi Srikanth, Thank you for the reply. To access the data is not the issue it can be either access using the “ExtractDataTable(0)(2).ToString” or as you said “Datatable.rows(i).item(j).ToString”. But over here the datatable structure gets vary depending on the data available in column.
Say I want to access data from “Column C” using the above syntax it can be accessed “Datatable.rows(0).item(2).ToString”. Now for another record column B value is not there in that case the datatable which is getting created using datascraping activity contains less column since column B doesn’t have value so now if I try to access value of column C using the syntax “Datatable.rows(0).item(2).ToString” it will return wrong result since now column C has shifted to position of column B.

  1. How to get the column headers along with the data from web table in datatable using data scraping activity.
  2. How to make the datatable structure consistent when data is not available for all the columns for all the records.