Automation freezes in data scraping

Hello,

My automation freezes and remains on the same screen for hours when there is no data available from that screen to scrape. I tried putting a condition if datatable = Nothing but that also did not solve the problem. The website I am trying to scrape from:
https://www.amazon.com/ask/questions/asin/B08DFPV5Y2/2/ref=ask_dp_iaw_ql_hza?isAnswered=true#question-Tx11J5DRGD6ZHYY
from here I scrape the question link which works fine

This is where I use the data scraping activity and it stops working when no one has answered the question:
https://www.amazon.com/ask/questions/Tx11SBKNL3QK4NV/ref=ask_ql_ql_al_hza

what I want is question, question date, question votes, all answers to question, answer hellpfullness votes, answer date, answer username. Here is my code:
the issues are:

  1. when a question has no answers the automation just freezes.

I have attached my code below. The excel file that I use as input has the product ASIN(product code) and a Link column which has the link to the main product page(not question answer.)
Ques Ans With Votes Final 24.xlsx (8.5 KB)

Main.xaml (29.7 KB)

  1. Here is what my automation does:
  2. It opens an excel file with product link and ASIN(product ID)
  3. Then from the ASIN number it goes to the main page for that product where it can find all the question and answers
  4. Then it extracts all the question links, votes, question
  5. It then goes on each question link and uses data scraping to get the answers, answer date, answer user name, answer helpfullness
  6. It uses screen scraping for getting the question date
  7. It then adds a column where it adds the question, votes, and question date for the above data(multiple answers) per question
  8. then it sets all the data tables to null
  9. Repeats the process all over again

you can try this
Main (1).xaml (39.1 KB)

i made a few small changes

image

  1. after opening QA link, check if “question doesnt have any answers” text exist, if yes just build an empty datatable, otherwise extract results
    image

– if theres no answers but you want to show a empty row (with question, date but no answers), you can uncomment
image

Hello,

I modified the tool to scrape multiple pages for each question and the solution no longer works and freezes again. Have attached the modified code below

Thanks

Main.xaml (43.2 KB)

@SSSS_MMMM @SSSS_MMMM because you commented out this
image

can you uncomment it??? or else it will get stuck when theres no answers…

if you commented it wont know if no answers exist

@jack.chan thank you

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.