Automation freezes in data scraping


My automation freezes and remains on the same screen for hours when there is no data available from that screen to scrape. I tried putting a condition if datatable = Nothing but that also did not solve the problem. The website I am trying to scrape from:
from here I scrape the question link which works fine

This is where I use the data scraping activity and it stops working when no one has answered the question:

what I want is question, question date, question votes, all answers to question, answer hellpfullness votes, answer date, answer username. Here is my code:
the issues are:

  1. when a question has no answers the automation just freezes.

I have attached my code below. The excel file that I use as input has the product ASIN(product code) and a Link column which has the link to the main product page(not question answer.)
Ques Ans With Votes Final 24.xlsx (8.5 KB)

Main.xaml (29.7 KB)

  1. Here is what my automation does:
  2. It opens an excel file with product link and ASIN(product ID)
  3. Then from the ASIN number it goes to the main page for that product where it can find all the question and answers
  4. Then it extracts all the question links, votes, question
  5. It then goes on each question link and uses data scraping to get the answers, answer date, answer user name, answer helpfullness
  6. It uses screen scraping for getting the question date
  7. It then adds a column where it adds the question, votes, and question date for the above data(multiple answers) per question
  8. then it sets all the data tables to null
  9. Repeats the process all over again

you can try this
Main (1).xaml (39.1 KB)

i made a few small changes


  1. after opening QA link, check if “question doesnt have any answers” text exist, if yes just build an empty datatable, otherwise extract results

– if theres no answers but you want to show a empty row (with question, date but no answers), you can uncomment


I modified the tool to scrape multiple pages for each question and the solution no longer works and freezes again. Have attached the modified code below


Main.xaml (43.2 KB)

@SSSS_MMMM @SSSS_MMMM because you commented out this

can you uncomment it??? or else it will get stuck when theres no answers…

if you commented it wont know if no answers exist

@jack.chan thank you

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.