Increasing wait time for response argument in dialogflow

Help Chatbots
1

I have a dialogflow intent mapped with an orchestrator process. The process is extracting data from multiple webpages and may take sometime (from as low as 10 minutes to as high as 2-3 hrs). Now the issue is that the chatbot shows me error “Something went wrong, please try again” after 5 minutes. Though the process continues to run and I get my output fine. But I’m not able to publish it on chatbot as response (as it appears the intent only waits for 5 minutes for the output argument) How can I increase this wait time? Would changing life span of context help?
Any help would be really appreciated.

2

Hi @manisha_mehta, as you’ve correctly observed, our service waits for up to 5 minutes for the job to complete before timing out (and showing you the ‘Something went wrong.’ message). This is a setting defined in the service, and there is no configuration that you can currently specify to change it. The reason we limit this to 5 minutes is because a wait much longer than this would lead to a poor chat experience for the end user.

As an alternative, it might be best to receive the request from the user via chat, have your mapped process trigger the process that extracts data (using the start job activity), and send the result to the user via email if using the web channel, or by posting a message as the Slack bot to the user using Slack APIs.

3

Hi @Karan, thanks for your response.

I’ve mapped the processes with dialogflow intents. There are 2 processes which get triggered by the training phrase/entities maintained in the intents. The first (shorter) process:

  1. logs into a website,
  2. based on a search criteria, shows search results (search criteria is provided by the end user through chatbot)
  3. scrapes the data from search result table and
  4. Adds the data to Orchestrator Queue (including direct url of each result item)

Now depending upon the number of pages of the result set, this process may take a little over 5 minutes (when there are 20 odd pages and 450 odd transactions). I’m actually sending an output parameter back to Slack API to show some meaningful message, though I’m also ok to send a message like “The process is taking a bit longer, blah blah” (if the process takes longer than 5 minutes), but it seems that I’m not even able to do that. As soon as the service sends “Something went wrong…”, the dialogflow intent expires and doesn’t show the response message defined in the intent.

There’s another longer process which

  1. gets the transaction items from the orchestrator queue(URLs)
  2. navigates to the url,
  3. fetches data from the webpage,
  4. stores data in an excel sheet
  5. sends an email with the sheet as attachment. This email id is provided by the end-user from chatbot

Since this is a consulting project, I thought chatbot is the best way to get user inputs. So I don’t want to replace it with other triggers.

Is it possible to either

  1. Make the wait time configurable? Or
  2. Let the developer configure/change “Something went wrong…” message? Or
  3. Change the message to “The process is taking longer, please wait…”?

I think #3 makes more sense, as the process continues to run in the background, so to show the message “Something went wrong…” to the end user is misleading.

Also attaching the screenshot of the chatbot for your reference:

image
image2186×1493 188 KB

Your help would be really appreciated.

Thanks
Manisha

4

Thanks for the detailed overview of what you’re trying to accomplish @manisha_mehta!
In this case, would it be possible to structure the processes such that:

  • Process 1 (triggered by the intent):
    1. runs a search on the website
    2. captures number of results (if available on the site)
    3. sets the output argument to a message like ‘There are count properties available in area.’
    4. adds the URL, the slack user ID to a queue for process 2 to pick up
      This ensures that your process finishes and the output is sent to the user based on the response you’ve defined.
  • Process 2
    1. navigate to the search results, scrape data from the search result table, etc.

Finally, once you’re ready to send out the email, you can use the Slack API to:

  1. get the users email from their Slack profile using their Slack user ID (instead of asking them for it)
  2. respond to them on Slack to let them know that a list has been emailed to them

Hope this helps! Happy to get on a call to talk through this scenario further :slightly_smiling_face:

5

Hi @Karan. This solution definitely looks promising. I’ll definitely try and will share how it goes.

Thanks a lot for help…