How not to click present file link, which is already downloaded and jump to next file link and click within for each loop

Hello Guys,

I am executing for each loop to download 500+ files , but in case any system error or exception occurred, then how to download only remaining files and not to run loop for previously executed files.
for ex:
start loop
1st downloaded previously
2 nd downloaded previously
|
|
120 th want to download

please help

HI @smita.mobifly

I would keep a small track file that includes the files that I downloaded. Most probably an excel would work.

So at the beginning of the process, we check whether the tracking excel do not have any data. If not, that means we are good and we can continue downloading. After each download, I would enter a record to the excel (let’s say the file name for example) so that it will maintain a list of all the files i downloaded. So that if a system error occurs, we can read the excel file and identify which files we downloaded before and contirnue from there.

Example
Check excel : empty
download 1st file
Update excel
download 2nd file
Update excel
– System crashes

Restart of the program
Check excel file : not empty

  • Read the data in the excel to a datatable and identify which files were downloaded
  • Then, if the next downloading file is not available in the excel, download, else, skip to the next

You can have this in a memory as a array or something, However, I wanted to make sure it also supports in case of a complete failure and the process also needs to start again. In that case, having it stored in somewhere like a excel file helps alot.

2 Likes

I would use queue for this task.
Put all links in queue; mark each item as completed when it successfully downloaded; get a new item while it is not empty

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.