I’m facing an issue while searching for data on certain websites through a UiPath bot. Some websites are able to detect that the request is coming from a bot, so instead of showing the search results, they reject the request and display an error page.
In such cases, what best practices should I follow while developing the automation in UiPath?
I’ve also noticed that some activities do not work reliably, such as:
Click activity
Send Hotkey (Enter)
Keyboard shortcuts for Enter
Could you please suggest the right approach to handle such scenarios and ensure the bot can successfully retrieve and process data from these websites?
First you should start with setting your Use Application/Browser activity to Hardware events (and the Click etc activities to “same as use app/browser”). The web page may be detecting that your automation is using Chromium, Simulate, etc and flagging that as a bot.
Yeah, stop doing it.
If the site is actively trying to block you, its because its against their TOS to do so. You should respect the sites terms and not try to bypass them, it can also get your account blocked on sites that require accounts.
You should get consent from the owner of sites to do automations on it.
They use these tactics since otherwise it can cost them alot of money cause bots can increase their network load and cause their sites to perform slower etc.