Adding The Searched Term to a Web-Scraped Table

Help
,
1

Hi There Community - I’ve been able to successfully google-search a list of terms from an excel file (using for each row activity) and save the results in a data table. However, how can I get each searched term to appear as a column alongside in my data table of search results so that I know which results pertain to which searched term?

Thanks!

2

Hi @aboldizsar004

So basically I believe the excel file has a list of URL’s which you access in the loop. Then you go to that page, get some data and store in a datatable.

I have one question here. How are you adding the data of the web site to your datatable? Is it coming through web scraping? Or are you getting the values to individual variables and then adding the data to the datatable using a Add Data Row activity?

3

Hi @Lahiru.Fernando

So here is my process so far:

  1. I created a list of keywords that I want to search in google within an excel file.
  2. I have the bot open up google within a browser.
  3. I then, using a “for each row” activity, have the bot input the keyword, get the first 20 results (using data scraping wizard), and output them to a data table.
  4. I then have the the bot repeat the process for each keyword within my excel file

Right now, my output data table only includes 2 columns: 1.) Search results and 2.) URL of each search result

I would like my output data table to include 3 columns: 1.) Searched Keyword (from my excel file) 2.) Search results 3.) URL of each search result.

Hope that clarifies!

4

Cool… now I understand the scenario…

I guess it will be easier to share a sample worflow with you on how to do this… Give me couple of minutes. .I’m working on a sample for you :slight_smile:

5

HI @aboldizsar004

So I have created a sample workflow for you. However, I didn’t actually do the data scraping part since you already know how to do it. But I added the extract structured data activity to simulate your data scraping. So if you are using my workflow, your data scraping should be added to that location :slight_smile:

So what happens here is this…

  1. Read Range to extract the keywords to a datatable
  2. Build another datatable with three columns to hold the final results
  3. Loop through the keyword datatable using For Each Row activity
  4. within the loop, do the google search and extract the data to a datatable
  5. Add a column to the datatable that was generated through the data scraping to hold the keyword
  6. Loop through the scraped data and add the keyword to the new column
  7. Merge the results to a final datatable that holds all the scraped data while it is ready to write the data to excel
  8. Then finally, write the data :slight_smile:

Note: Since I haven’t done the scraping, it has a error there… So don’t worry about it much… I just want to show the workflow on how it should be done :slight_smile:

LoopingThroughAndAddingColumns.xaml (15.4 KB)

Let know whether this works for you…

6

Hi @Lahiru.Fernando

Once my “for each row” loops through once on the extracted data, I’m getting an error on the “Add Column” activity saying that the column named “Keyword” already belongs to the data table. Any idea how I can go about resolving?

Appreciate your help on this!

7

Hi @aboldizsar004

The error says the column name is already available because it cannot have two columns with the same name. Make sure you add it only once :slight_smile: