How to add new data table to exist data table?

[greenlee]

Hi,

In the For Each Row in Data Table activity, one data table (dt_Extract) was obtained for each web page by scraping data. How to add a new data table for each new web page to dt_Result data table?

[Palaniyappan (Forum Leader)]

Hi
Did we try with MERGE DATATABLE activity
Make sure the column names, column order are same

Cheers @greenlee

[greenlee]

Thank you @Palaniyappan ,
I have encountered the following error in debugging at Merge Data Table activity.

Merge Data Table: Object reference not set to an instance of an object.

The dt_wordResult data table was blank initially. Does this cause any problem?

[Palaniyappan (Forum Leader)]

@greenlee

The dt_wordResult data table was blank initially. Does this cause any problem?

Yeah that’s right
Pls make sure you have atleast the column structure in that datatable

Either you can create with build datatable activity or
use a assign activity like this

dt_wordResult = dt_word.Clone

Then use a clear datatable activity and pass the dt_wordResult as input

Cheers @greenlee

[postwick (Paul)]

Are you reading data from multiple different web pages, or is it one web page with “next page / previous page” buttons for a list of data?

[Palaniyappan (Forum Leader)]

@greenlee

was obtained for each web page by scraping data.

If the web page has a next button make sure that it is selected while data extraction is performed
For more reference
In that case you don’t need to merge them separately with merge datatable activity
4. Indicate the Next button in the web page, application or document (if the information you want to extract spans multiple pages).

About Data Scraping

If it’s from different page then we can use merge datatable activity provided they all have same column structure

Cheers @greenlee

[greenlee]

Thank you @Palaniyappan ,

The web page I want to extract data from has an arrow button and a numeric number. However, when the arrow button is pressed, it moves to the page that has been skipped a few pages instead of the next page. So, I used For Each Row in Data Table activity by extracting different urls as shown below.

http://www.lampcook.com/food/food_dic_koen.php?search_mode=0&alpha_no=0&sql_one=&pagenum=1
http://www.lampcook.com/food/food_dic_koen.php?search_mode=0&alpha_no=0&sql_one=&pagenum=2
http://www.lampcook.com/food/food_dic_koen.php?search_mode=0&alpha_no=0&sql_one=&pagenum=3
…

I am thanks also @postwick for our comment.

[postwick (Paul)]

Yeah sometimes the scraping doesn’t handle the next/previous properly - depends on the web page. You shouldn’t be using For Each Row. You should set up a While that extracts the first page of data into a temp DT then merges it into what will be the final DT. Then you look for whether or not there is another page to process (Element Exists), and if there is then browse to that page and repeat the steps. Your true/false from Element Exists will be the condition of the While.