Append range keeps writing same data

Hi Guys,

I know this topic has been raised up a few times already, but I still can’t find the solution.

MAIN TASK is:
1 To crap data from the web from multiple pages (I limited it as 4 pages for now).
2 Merge all data from multiple pages into 1 Excel file. (So every loop new data will be added to the previous one)

My WORKFLOW:
1 I created WHILE loop
2 Then, by using Data Scraping Wizard I identified what data to scrap (when it asked me about multiple pages, I said NO)
3 All scraped data then goes to ExtractDataTable1 variable
4 Data gets saved by Append Range activity to Excel file
5 Clear Data Table (ExtractDataTable1)
6 Then is scrolls down to the bottom of the page
7 Press down page 2,3,4 etc.
(looping whole process again)

THE PROBLEM is, in resulting Excel file, I get same data only from Page1 to be saved repeatedly right after another, so If I scrap four pages, in result I get 100 records (25 from each page), but they just repeat each other. What I want to achieve is to have all data from four pages added one after another in one Excel file.

For reference, I put Write Line activity (ExtractDataTable1.Rows.Count.Tostring) to check if the DataTable get cleared to null or not. It returns nothing.

I have attached my workflow screenshot, please help me to figure out what I am doing wrong.

Thanks in advance.

You should indicate the next link selector property.(Button to go to next page)
image

Thank you for reply, but the thing is, there is no Next page button at the webpage, only 1234 …
So, if I say YES and click on 2 when Data Scrap Wizard is asking me about multiple pages, it will be an error, since it works only when you have “Next page” or > buttons.

Can you show button page1 selector? In while loop activity you should declare index variable, increase + 1 and put it into your next page selector after each loop.

If I uderstand you correctly it should be in “Click next page” step in my workflow. This is what is inside.
webctrl aaname=’"+counter.ToString+"’ parentid=‘searchResultData’ tag=‘A’ />

FYI, the webpage is this - “https://www.timesjobs.com/candidate/job-search.html?from=submit&actualTxtKeywords=.net%20Developer&searchBy=0&rdoOperator=OR&searchType=personalizedSearch&luceneResultSize=25&postWeek=60&txtKeywords=0DQT0net%20developer0DQT0&pDate=I&sequence=1&startPage=1

Yes. Exactly!.

You mean I should type this code:

webctrl aaname=’"+counter.ToString+"’ parentid=‘searchResultData’ tag=‘A’ />

To NextLinkSelector in DataScrap activity? Like u mentioned the screenshot before?image

Check attach file:
RecordingSequence.xaml (12.1 KB)

You need dynamic click activity to go to next page as below:

I have this selector set in “Click next page” actually. So, it goes without errors clicking all 23456 etc. buttons and scraping data and saving excel file.

And I get data in Excel file, but it’s only from 1st page and keeps repeating.

So, I think, it has something to do with:
1 Clear Data Table - so it doesn’t clear it, and keeps copying 1st page in Apend Range.
or maybe
2 Something wrong with variables / DataTables, maybe I should create one more variable for DataTable?
3 Or something is wrond with DataScrap activityitself, it’s probably scraping 1st page (because I showed it in Wizard, but then because I didn’t click multiple pages, it keeps scraping only 1st page), but then why it doesn’t scrap other pages altought the browser moves to 2345 pages??

I do no… ;(

So, I did this experiment - I’ve deleted Clear Data Table step, then ran it and suprisingly instead of 100 records (4 pages), it showed me 150! So, I guess it’s a result that stayed in the memory from previous session? so ClearDataTable actually is working… so probably something is wrond with DataScrap, since it doesn’t scrap other pages???

So, I tried to add the line below to NextLinkSelector in DataScrap activity:

webctrl aaname=’*’ parentid=‘searchResultData’ tag=‘A’

now actually it goes through all pages, scraps all data, BUT the result have changed - now it shows only last scraped page in Excel. (I tried to delete ClearDataTable, but then it starts repeating again)

so I guess he solution is somewhere in the middle??

anyone, any ideas?