ML Skill GPU And Advanced Infra Settings

News Knowledge Base
, , ,
1

Why are Advanced Infra Settings needed?

About:

When configuring an ML Skill, users often have questions about why they would need to use GPU or the Advanced Infra Settings for ML Skills. For some users, using the base configuration will be sufficient, while for other customers, adding GPU or increasing Replicas or Resources may be necessary.

The requirement is dependent on the volume of documents being processed, the number of Robots simultaneously using the skill, the size of documents being processed, etc. When an ML Skill becomes overwhelmed with requests, errors such as "ML Skill Unavailable - 503" may be observed. If this occurs, the skill may need to be modified to use GPU or Advanced Infra settings. This guide will walk through various scenarios when adjustments may be required to ensure the most efficient processing.


GPU:

Enabling GPU gives the skill the ability to perform extraction much faster than when using CPU. When GPU is enabled for a skill, more AI Units will be consumed per hour as long as the skill is available with this option enabled. However, if a large batch is expected to be processed, enabling GPU temporarily may be more cost-effective as the extraction for each document would complete much faster vs processing on CPU alone. After the large batch of processing has been completed, GPU could be disabled if desired.

Advanced Infra Settings:

Replica Count - A replica is an instance of a model. More replicas lead to more instances of the same model so you can process more pages. Use this to process more pages in parallel if the model is called several times in parallel. Increasing the replica count for the ML Skill makes the skill Highly Available (HA). When the skill will be used for concurrent usage (multiple robots using the skill simultaneously), or when business-critical processes require the skill to be highly available for processing, it is a good rule of thumb to increase the replica count. Note there is no "magic" number here, so the appropriate replica count may require some trial and error. If 1 Replica is chosen, High Availability (HA) will be broken.

Resources Per Replica:

If GPU is not enabled, the option to increase Resources Per Replica (CPU + RAM) modification is available in the Advanced Infra Settings. Increasing the CPU + RAM is not as "powerful" as using GPU, but the AI Units cost is less than GPU. Again, there is no "magic" number here, so the appropriate resource configuration may require some trial and error.

Additional Notes:

  • For ML Skills deployed as Standalone AI Center or via Automation Suite deployed AI Center, no additional AI Units are consumed as the increased resources are dependent upon the customer's infrastructure.
  • For Skills hosted in UiPath Cloud, more AI Units are consumed when GPU or anything beyond the basic Infra settings are configured for a skill as these settings require increased hardware in UiPath Cloud.
  • For more information regarding AI Unit Hardware Cost, see the document - AI Unit Hardware Cost.
  • Advanced Infra Settings are only available if the AI Units licensing model is utilized.