Hi guys
I have some issues with one of my projects and I was kinda hoping that the community coiuld point in the right direction. It’s about an online agenda for doctors, for each week it’s suposed to scrape the appointments and patientdetails and write it to a csv file, the scrapping used to work great, but I changed a couple of thing and now it’s verry unpredictable. The mainissue I have with it, I’m using a config.json to get certain vallues that are needed in the program, like the startdate and the enddate of the scrapping period, also at a certain date the appointments are being archived, when it reaches that point, it should click the archivebutton and keep scraping there untill it reaches the end date. I maded a function to check the date of the current scraping with the expected date and use that same function 3 times with different dates, but only with the archive date it seems to work, so the program never stops, when it reaches the enddate it just starts over again. For thez scraping, as I said that used to work but now sometimes it does sometime it dosn’t, I’ll include a video to demonstrate the flow. The description is, it opens a browser with a certain url it get’s from the jsonfile then it goes to the printpage to scrap the appointments for each day of a week, of the first scrapping I only need a date, time, first- and lastname, I use the printpage of the way the data is displayed, I tried to use scrapedate but everything is on one row and that maded the formatting to hard, each day I merge in a dt and at the end of the week I write it to a csvfile. Then I navigate to the startdate and click on ‘vertegenwoordiger’ (salereps) scrape all the salesrep appointments, go to consultatie do the same and again for dringend, there kept in different lists, on the background I then have a function that filters all the real appointments from the not important onces (like vrij of beluurtje) then should it for every appointment click on the corresponding field scrape the popup and if it’s a patient click on the profile button and scrape the missing information of the profile page, at the end of the week everything is written to csv, patients and salesreps are kept sepperateDoctena_SanMax.zip (52.8 KB) - YouTube