Need some help to for data scrapping from salesforce report, which is dynamic(number of cases change).
Issue is when i am using Data scrapping wizard, fetching all columns and report is on single page, but i do need to scroll down to see new data rows- i am getting only certain number of rows like 26 or 27. But it’s 291 records in total and as i said it will keep on changing.
Attaching screen shot of activities:
How i can read all the rows of table?
Any help appreciated!!
So you mean to say that’s a responsive website where the new records get loaded when you scroll ?
If that is the case, then you need to add
Data Scraping in a loop !
Thanks for your response!
Yes on scroll new records get loaded.
By loop, i am trying something like this:
In a do-while loop:
- Data scrapping wizard in data table dt1
- write dt1 in excel sheet
- Mouse scroll activity
Loop through above steps for certain number of times.
Results i am getting in excel sheet got duplicate rows and some blank rows
Then using Remove duplicate rows to clean up the excel.
I couldn’t find any other way. Is there any other way possible?
I used the same strategy as @Nithinkrishna for a similar website where more rows are displayed when you scroll down.
You keep scrolling down and extracting data in a loop until previous extraction rowcount matches the current extraction rowcount. Here is a more detailed explanation:
- Initialize ExtractDataTable variable using Assign activity. (you need to use the same variable name here and in ‘Extract Table Data’ activity) ExtractDataTable = New DataTable
- In a Do While Loop:
a. Use Assign activity to copy data to a new datatable variable. PreviousTable = ExtractDataTable.Copy
b. Scroll down the page using mouse scroll or PgDn key
c. Scrape data to ExtractDataTable variable
d. Set condition for Do-While activity to Not (ExtractDataTable.Rows.Count = PreviousTable.Rows.Count)
- Write to excel or perform other actions you need on the datatable.
Hope this helped!
This method looks cool. Only change on my thought is to page scroll based on last row logic (But that would be a bit complicated as well)
So for now this looks fine.
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.