Understanding Delayed Workflow Execution and Extended Waiting Time Beyond Configured Timeout in ML Extractor Activity

What Causes Workflow Execution to Be Blocked and Waiting Time to Exceed the Configured Timeout in the ML Extractor Activity?

Issue Description

When executing a Document Understanding workflow that includes the ML Extractor Activity, the workflow may encounter scenarios where the ML Extractor fails due to certain issues. In such cases, the activity remains blocked, exceeding the configured timeout significantly.

Root Cause

One common cause of failure is a network-related issue, such as attempting to upload a PDF file that is too large, resulting in the request failing. For example, an error message might appear as follows:

SendRequest() failed: System.IO.IOException Failed to write data to the transport connection: The remote host forcibly closed an existing connection. < The remote host has forcibly closed an existing connection.

Additionally, if the RetryOnFailure property of the ML Extractor Activity is enabled, the activity will automatically retry failed requests even if no RetryScope activity is defined within the workflow.

Resolution

Disabling the RetryOnFailure property setting in ML Extractor Activity will make the block and waiting time to be expected and close to configured Timeout value.

The details of the property can be found here:

  • RetryOnFailure - Automatically retries the machine learning model execution to eliminate transient network errors. If checked, the activity retries the execution.