What is good my dudes. Back like I never left with more questions.
My project contains a process in which I open up a new browser to a URL i get from a data table generated in a previous process. Upon reaching the site, I have a FindElement in place to look for a table I want to scrape. In summary, open browser -> find element (data table) -> scrape table. If the find element returns false, no table is scraped and I move on with the process.
Now until about 4 days ago, this was running smoothly, but now I’m faced with an issue. At times, the page will open up and just stay there loading for long periods of time. Usually it opens up and reads the table within a few seconds, but there have now been 3 occurrences where it takes a little longer.
What I’m trying to figure out is how to put in place a method that checks how long the page was open and refreshes it to try and load it faster. This is so the bot won’t time out in the middle of the most time consuming part of the entire process.
To try and simplify it, After the page opens the first time, if it takes too long to find the element (table), I want the bot to refresh the page and look for the element again until it finds it relatively quickly. The only real problem I’m trying to combat here is timing out because the page took too long to load.
I can try to elaborate a little more if necessary, but if anyone has any suggestions that would be awesome. Every question I’ve asked has helped me tackle an obstacle I’ve faced in this project and I couldn’t be more grateful of the input I’ve received, so thank you.