Adding a column to the Data Table after a data scrape

I have a For Each loop which takes a search term value such as “Ui Path Developer” (saved in a variable called ‘template’) and searches the LinkedIn job page. Then I scrape the results of matching jobs into a data table and export it out to an Excel.

I’d like to add a column so I can identify which search term corresponds to the result sets. Example below. I tried using Add column activity after the scrape but got errors. Any suggestions?

Current behavior:
Capture

Hi!

After extracting the data to the datatable, before doing the For Each, you could use Add Data Column activity to add the desired column to the data table.

Then inside the for each, to populate the new column you need an assign activity like this: row("NewColumnName") = [TheValueYouWant].
After that, when you write the data table to excel it should contain what you’re looking for.

@sofsheikh

1.Before For each ,create a Build Data table and add the columns you want and store it in a Data table Dt.
2.use for each row
3.use Add data row and assign the scarped data variables in list of array to the data table Dt.

Regards,
A Manohar

Thank you for the responses. I am not understanding the directions but it may be because this is my first time trying this topic. I tried following the steps you two provided but I’m doing something wrong. I’ve attached my workflow for your review. What am I doing wrong?

Main.xaml (53.0 KB)

You want to add your new column after you run Extract Structured Data, because that will overwrite your datatable…

Nope, it did not work adding the add column after the extractdatatable. Any other suggestions?

Hi,
Were you able to look at the sequence I shared to see what I might not be understanding?

Hi! Sorry for the delay.

As @bcorrea said, ExtractDataTable variable is overwritten every time Extract Structured Data is executed. You need to add the results of each Extract Structured Data in another table that isn’t overwritten.

You can achieve this by creating a new DataTable variable at a higher level, and after each Extract Structured Data use Merge Data Table activity to add the newly extracted table to that variable.

could you attach workflow for this? I have been all day trying to do it but no luck. Where I create the new Data Table variable? in Assign activity?

Hi, welcome to the community!
Do you have a sample of what you are trying to do in your project?

Hi there. Thanks for your prompt response. I have a links.xlsx file with a ”Reference “ column and a “links” column with 3000 URL approx… I created a robot to scrape the content (table format) on those links and save it in an excel file but I don’t know how to add the ”Reference “ column into the results table. I get an error message saying it is already added when I execute the “For each” activity.ScreMain.xaml|attachment (17.7 KB) enshot (20)

And my smaple workflow Main.xaml (17.7 KB)

That is because you are adding the new column inside your ForEach, so it will try to add a column with the same name more than once… If you can, just start your excel with all needed columns already there and only add the values for each new row… If you cant, than make sure you dont run the Add Column activity more than once for the same new column you will need.

Thanks but I will need a little bit of more detail, I am confused.com:
1 - If I can: what activities do I need to add the value for each new row? Can I use a "For each row activity within the excel scope and assign each scraped data column to a variable? how? Will the “Ref” cell corresponding to that row be placed in the correct row?

2 - If I cannot: how I dont run the Add Column activity more than once?
I have been up and down through the courses and the forum and I cannot find the solution. Would you be so kind to provide more detail? A sample workflow will, of course, be a dream come true

See if this helps: Main.xaml (21.7 KB)

Fantastic, it works like a charm! many thanks for your help

1 Like