Next link selector in extract structure data activity always wait whole timeout period, if link doesn't exists on web page


I am using Extract structured data activity ( with NextLinkSelector option. If the link doesn’t exist on web page, activity always waits until the timeout period is not reached (30 seconds by default). Here is the definition for NextLinkSelector: . How can I achieve that activity continue immediately, if link doesn’t exists on web page. I am facing the same issue with Element exists activity.

Thanks for your help.

@Gregor_Kocevar Have you tested to set the Timeout to something small? 1000 ms = 1 second


1 Like

Yes, I tested with a timeout (1 s) and everything is fine. What I am worried about is that the activity will end too quickly (I mean, the activity will not extract all data on the web page). My activity is actually searching on google for results on the first 10 pages (sometimes there is less than 10 pages and in that case, everything waits).

I guess there’s always that possibility if the browser is slow to render the Next Link element for some reason. While I do think 30 seconds is overkill, a 5 seconds timeout might be a more reasonable compromise if your bot misses a lot of results because of the 1 second timeout.

1 Like