i am using data Scraping activity to extract the data from the table it is extracting 4 record of table correctly, but when in the table if there is more record like 28 record it is extracting only the 8 records so can anyone let me know how to extract the entire table at a time and write it to the excel.
Can you please make sure couple of things,
- Set the MaxNumberOfResults to 0
2.Set DelayBetweenPagesMS if you have Next Link Selector to extract data from next page.
Hope in the preview window you can able to see few extracted data correctly as below
Note: Please place the code inside try-catch block and log the exception.message as well to identify if there is any internal errors.
Usually in Extract Datatable activity default value for number of records to be fetched is 100
As yours is 27 by default it has to be picked provided you haven’t changed that property value
We can keep 0 if we don’t know how many rows will be there and also if we want to get all data we keep as 0
- This can be first validated from Preview stage itself
That is once after scrapping you can see all the data as a preview and there make sure you have all the records
If not check where we are missing with the steps as per this doc
- And as @Jobin_Joy suggested increase the delay between pages if you are trying to fetch from multiple pages