Creating a headless browser automation for website scraping

[19learningcurve]

Hi, I need to develop an automation where I have to navigate to multiple webpages, provide some input in text boxes, click submit button and then extract the available information.
Can I use headless browser automation for above case or else I have to go with conventional approach of UI Automation?

[ashokkarale]

@19learningcurve,

It is possible. Try this approach discussed here.

Is there is a way to scrape a website without opening in a browser?

I got it resolved this issue for my need. I wanted to open a headless browser and grab a string on the page. I used “use applicaiton/browser” activity. In the properties, I set the option browser webdriver mode to headless. Then I launched the website and indicated the website to the “use app/brow” activity. Then I used the gettext activity and capture the elements of my string. all the while no webpage was launched