Data scrape not working

Help
, , ,
1

Hello all,

I have the Community Edition.

I am using Data Scraping and I’ve used it successfully in the past but now it doesn’t seem to be scraping.
Here is the image of the Scrape activity
2019-12-05_160916
Here is the Extract Structured Data ‘TABLE’ piece

2019-12-05_160857
2019-12-05_1608571922×158 72 KB

Here is the Scope image
2019-12-05_161105
and the Extract Data process image
2019-12-05_161143
2019-12-05_161143513×522 69 KB

What have I missed?

Thank you in advance

2

Hi
Are we getting any error
Or if it’s not taking the data itself then kindly try once after re scrapping
Cheers @mworth123

3

@Palaniyappan - No error, I am dumping it into an Excel table and the table is always blank. Can’t figure out what Im missing

4

Fine let’s go steps by step
—once after getting the datatable from data scrapping pass that variable as input to write line activity and mention like this
Datatablevariable.Rows.Count.ToString

If that shows any record number then we are good, data is scrapped
If not we need to check with the selector if the data scrapping activity and re do the scrapping process
Or

If that has rows in it means then we need to check whether the variable passed has global scope in e variable panel and the right variable is passed to the excel file
Mostly this part of validation will be fine already
I hope we would be probably concentrating on the first validation

Cheers @mworth123

1 Like
5

Hi, data scraping can go wrong because of many reasons , could be because of selector , or corrupted metadata or missing mapping of the datatable variable

At extract data activity , Please set continue on error to false

And run again, this will give you error , because of what reason you are getting the error

If there is no error thrown and unable to write the data in the excel file
Now go to extract metadata property and check whether any idx attributes have generated

If that doesn’t work , now use output datatable activity
To write the extracted Datatable data into string and use writeline to see the data

6

Hi @mworth123

Have you use write range to pass the data?
If not use write range activity from the workbook and pass the output datatable GEMAtable.And dont forget to enable the add headers

Cool,
Regards,
Gulshiyaa

7

@gulshiyaa - Thank you, yes Im using a Write Range activity to save the scraped data table as follows:
2019-12-09_172039
and I am using Add Headers
I can’t get the Properties sections to load.

Is it possible to restart and redo the Data Scraping section or do I need to delete and start over?

8

hey remove the “A1” and keep it as empty " "

9

@gulshiyaa - Thank you, Ive done that but haven’t had time yet to test it

10

Hi , @vinay_reddy… I have did all the above mentioned things, Set ContinueonError to False, Checked idx in metadata and i have edited it.And my selector is also in green and shown validated. Yet, iam not able to extract the data. It is showing empty data table.

11

Hi, I’m facing the exact same problem as you and wanted to ask if you have resolved it?

12

Hi, I am also facing the same issue. Any guidance is available to troubleshoot? No errors. But DataTable is empty. During “Design Time” scrapping is working. Found a dyanmic ID of an element and fine tuned it. A random 13 digit number is replaced with “???”. Validation is successful. But the DataTable is still empty.