Data Scrape Cache?

Good morning. I have an automation that scrapes data from an on prem accounting system. I gather two columns from this data table, Item number and cost amount. The goal of this is to determine if any sub components of the parent item have not been costed and therefore have a null value in the total. I scrape, filter the dt, apply finalTbl.Rows.Cast(Of DataRow)().Where(Function(row) row.ItemArray.Any(Function(field) (TypeOf field Is System.DBNull))).Count() to the dt and then if the count is greater than 0 then the automation takes a screenshot and sends it to configuration management. The issue is, even though I apply a clear data table activity afterwards, the next item page that is scraped exports the previous item data as well as the new item data. So, if the new scrape has no blanks and the old one did, it takes a screenshot in error. Why is the dt not clearing between scrapes? Any ideas on how to solve this?

Hi @ Chris_Bolin,

Use Activity clear dataTable after perform your wrok. And your problem can be in the table element can be diferent can you check and becouse of that you always get the same information on your DT.

1 Like

hi @Chris_Bolin, do you remember to Clear/ set to new to your DataTable?

Thanks Luis. I cannot think of a scenario where the table element would be different. The automation goes back to the home screen of the application after one item process is completed and enters a new part number. So, the part number data on the page being scraped is exclusive to the new part number. I can only think that it is being stored in a cache or the dt that is extracted from the scrape is not clearing using the activity Clear DataTable.

I do utilize the activity Clear DataTable but it does not seem to clear

I use clear data table…how do I “set to new”

Is clear datatable outside the for each ?

I do not have a for each in this automation. There are prior validation automations that take place against the part number before the data scrape. This flow is the final validation. When this is done, it removes the line item from the working list and goes to the beginning of the entire process and reads the second line item based upon a flow decision - workingFile row count>0

But is your clear table outside the scrap data ?

I tried it outside of the scrape in the x-out sequence at the end and then thought last night on my commute home that perhaps it needs to be inside the scrape. It does the same thing in either situation


Thanks Luis…let me try that.

1 Like

Luis, thank you for the help. This solution worked!

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.