Data Scraping tool doesn't work if MaxNumberOfResults set to 0

Help
1

I used the Data Scraping wizard to extract data from a table on a webpage. I chose the “extract all data” option when using the wizard and indicated the button for next page to get more data. There are about 6 pages of data with 2000 rows of data total. If I just choose some number like 100 for Max results, the data scraping happens just fine, but if I keep Max results at 0 (and I have ContinuesOnError set to False) then I get this error:

“Message: Input string was not in a correct format.
Exception Type: System.FormatException”

I’m ultimately trying to save the datatable to a CSV and if Max results isn’t set to 0 I get a perfect CSV, but when set to 0 (and ContinuesOnError for data scraping set to True) I would get an error about the datatable not referencing an object and the CSV file that was output was just a couple random symbols… no table of data.

One last complication to this whole mess, if I don’t choose “extract all data”, when first using the data scraping wizard, and I instead choose the columns myself (indicating first and second element), then when I press “Finish” in the wizard it doesn’t even ask if there are multiple pages, it just closes the wizard and does not leave behind any activity for the data scraping…

2

Hi @skarbrevik

To make sure that you don’t try to save an empty Data Table to a CSV file, you can either check if the variable was initialized:
IsNothing(yourDT) - this will return a boolean
and then check the rows of the Data Table to see if all is correct.

As to the last issue, I have just tested it and it did display it for me.

Could you let us know the version of your Studio and of the activity packages (screenshot from the package manager)?