Data Scraping Taking time

Hi All, I am inputing splunk results to excel using Datascraping results → Excel Application Scope → Write Stream but during run time it is take 2 mins how to make it work faster friends?

Hi @Sathish_Ashokan

How much data does the data scrapping result hold?

Also, please clarify if starting from data scrapping and until writing it to an excel is taking 2 mins in total ?

100 page datascraping and excel application scope

Hi @Sathish_Ashokan

Assuming 10records in each page then for 100 page it will be 1000 records and having this data 2 mins seems to be an ideal time to me :slightly_smiling_face:

However, you can try to use simulate method if not already used for more efficiency.

Cheers

i mensioned 100 pages but actual result only 2 records but it is taking time people asking why there is delay. less than 10 records why it is taking so long to move to next sequence anything i missed?

@Sathish_Ashokan can you share the screen shot ?

Have you used simulate ?

Have you used modern design in your project ?

@Sathish_Ashokan

Try using the activities in the below order and check if it improves the performance.

  1. Excel Application Scope
  2. AttachWindow
  3. Extract Datatable
  4. Write Range

In your current workflow, you are trying to open and close the excel application every time after you write the data table to the excel.

Check if moving the excel application scope outside improves your performance.

Hope it helps!

Hello @Sathish_Ashokan
Try Below:

  1. Use time out Properties in each web activity like Attach browser, Data scrapping etc. Set it as 2000
  2. After data extraction, Use the write range workbook activity instead of the Excel application scope.
  3. Check the necessary Delay Activities with specific time delays

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.