Getting issues with Data Scrapping from Salesforce report - Not able to fetch all rows

Help Studio
1

Hi Everyone,

Need some help to for data scrapping from salesforce report, which is dynamic(number of cases change).
Issue is when i am using Data scrapping wizard, fetching all columns and report is on single page, but i do need to scroll down to see new data rows- i am getting only certain number of rows like 26 or 27. But it’s 291 records in total and as i said it will keep on changing.

Attaching screen shot of activities:

image
image1603×963 73.6 KB

How i can read all the rows of table?

Any help appreciated!!

Thanks,
Deepika

2

Hey @deepika.garg

So you mean to say that’s a responsive website where the new records get loaded when you scroll ?

If that is the case, then you need to add Data Scraping in a loop !

Thanks
#nK

1 Like
3

Hi @Nithinkrishna,

Thanks for your response!

Yes on scroll new records get loaded.

By loop, i am trying something like this:
In a do-while loop:

  1. Data scrapping wizard in data table dt1
  2. write dt1 in excel sheet
  3. Mouse scroll activity
    Loop through above steps for certain number of times.

Results i am getting in excel sheet got duplicate rows and some blank rows

Then using Remove duplicate rows to clean up the excel.

I couldn’t find any other way. Is there any other way possible?

Thanks!

4

I used the same strategy as @Nithinkrishna for a similar website where more rows are displayed when you scroll down.
You keep scrolling down and extracting data in a loop until previous extraction rowcount matches the current extraction rowcount. Here is a more detailed explanation:

  1. Initialize ExtractDataTable variable using Assign activity. (you need to use the same variable name here and in ‘Extract Table Data’ activity) ExtractDataTable = New DataTable
  2. In a Do While Loop:
    a. Use Assign activity to copy data to a new datatable variable. PreviousTable = ExtractDataTable.Copy
    b. Scroll down the page using mouse scroll or PgDn key
    c. Scrape data to ExtractDataTable variable
    d. Set condition for Do-While activity to Not (ExtractDataTable.Rows.Count = PreviousTable.Rows.Count)
  3. Write to excel or perform other actions you need on the datatable.

Hope this helped!

1 Like
5

Hi @deepika.garg

This method looks cool. Only change on my thought is to page scroll based on last row logic (But that would be a bit complicated as well)

So for now this looks fine.

Thanks
#nK

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.