RROR: Training failed for pipeline type: TRAIN_ONLY, error: malformed node or string

Feedback AI Center
, ,
1

I am trying to use a train-only pipeline run to train some data with the MultiLabelTextClassification model. But the following error log was produced:

ERROR: Training failed for pipeline type: TRAIN_ONLY, error: malformed node or string: <ast.Name object at 0x7f1327022d30>

How do I interpret this and debug? If it helps, here is the full log:

Train only of MultiLabelPack 23.5.0.0 launched - Run 96b8db6f-1e66-44e4-90a4-a6bbb3be3bc9
Train only of MultiLabelPack 23.5.0.0 started - Run 96b8db6f-1e66-44e4-90a4-a6bbb3be3bc9
Train only of MultiLabelPack 23.5.0.0 scheduled - Run 96b8db6f-1e66-44e4-90a4-a6bbb3be3bc9
Train only of MultiLabelPack 23.5.0.0 failed - Run 96b8db6f-1e66-44e4-90a4-a6bbb3be3bc9

Error Details : Pipeline failed due to ML Package Issue

2023-08-08 04:01:05,232 - UiPath_core.trainer_run:main:74 - INFO: Starting training job…
2023-08-08 04:01:07,353 - UiPath_core.storage.azure_storage_client:download:118 - INFO: Dataset from bucket folder training-a0394d57-9d8b-4ee7-9b15-97c89266114e/cdd1a17c-7e4d-46ef-ad79-4c48ab6eac9e/94a4d594-4772-4c95-80ae-7a47186275bb with size 1 downloaded successfully
2023-08-08 04:01:07,353 - UiPath_core.training_plugin:train_model:130 - INFO: Start model training…
2023-08-08 04:01:07,353 - UiPath_core.training_plugin:initialize_model:124 - INFO: Start model initialization…
2023-08-08 04:01:07,354 - root:init:35 - INFO: Using options: {‘cuda’: True, ‘dataset’: {‘path’: ‘/microservice/dataset’, ‘csv_name’: None, ‘text_column_name’: ‘Description’, ‘target_column_name’: ‘Subcategory’, ‘delimiter’: ‘,’, ‘train_split’: 0.9, ‘encoding’: ‘utf-8’, ‘dataloader_workers’: 0}, ‘model’: {‘name’: ‘test’, ‘epochs’: 10, ‘tqdm’: True, ‘batch_size’: 24, ‘optimizer’: {‘transformer’: True}, ‘scheduler’: {‘type’: ‘hf’, ‘parameters’: {‘name’: ‘linear’}}, ‘gradient_accumulation_steps’: 1, ‘metrics’: [‘f1’, ‘ECC’], ‘path’: ‘/microservice/models/test’}, ‘network’: {‘name’: ‘roberta-base’, ‘freeze’: True}, ‘split_and_eval’: False}
2023-08-08 04:01:07,355 - UiPath_core.training_plugin:initialize_model:127 - INFO: Model initialized successfully
2023-08-08 04:01:07,355 - root:read_csv:103 - INFO: Loading data from /microservice/dataset/RogersTrainingData No 0X92.csv
2023-08-08 04:01:07,360 - UiPath_core.training_plugin:model_run:179 - ERROR: Training failed for pipeline type: TRAIN_ONLY, error: malformed node or string: <ast.Name object at 0x7f1327022d30>
2023-08-08 04:01:07,360 - UiPath_core.trainer_run:main:91 - ERROR: Training Job failed, error: malformed node or string: <ast.Name object at 0x7f1327022d30>
Traceback (most recent call last):
File “/home/aicenter/.local/lib/python3.9/site-packages/UiPath_core/trainer_run.py”, line 86, in main
wrapper.run()
File “/microservice/training_wrapper.py”, line 58, in run
return self.training_plugin.model_run()
File “/home/aicenter/.local/lib/python3.9/site-packages/UiPath_core/training_plugin.py”, line 195, in model_run
raise ex
File “/home/aicenter/.local/lib/python3.9/site-packages/UiPath_core/training_plugin.py”, line 171, in model_run
self.run_train_only()
File “/home/aicenter/.local/lib/python3.9/site-packages/UiPath_core/training_plugin.py”, line 255, in run_train_only
self.train_model(self.local_dataset_directory)
File “/home/aicenter/.local/lib/python3.9/site-packages/UiPath_core/training_plugin.py”, line 132, in train_model
response = self.model.train(directory)
File “/microservice/train.py”, line 42, in train
self.process_data(split=True)
File “/microservice/train.py”, line 87, in process_data
dfs = [self.read_csv(file_path) for file_path in csv_files]
File “/microservice/train.py”, line 87, in
dfs = [self.read_csv(file_path) for file_path in csv_files]
File “/microservice/train.py”, line 111, in read_csv
df[target_col] = df[target_col].apply(ast.literal_eval)
File “/home/aicenter/.local/lib/python3.9/site-packages/pandas/core/series.py”, line 4626, in apply
return SeriesApply(self, func, convert_dtype, args, kwargs).apply()
File “/home/aicenter/.local/lib/python3.9/site-packages/pandas/core/apply.py”, line 1025, in apply
return self.apply_standard()
File “/home/aicenter/.local/lib/python3.9/site-packages/pandas/core/apply.py”, line 1076, in apply_standard
mapped = lib.map_infer(
File “pandas/_libs/lib.pyx”, line 2834, in pandas._libs.lib.map_infer
File “/usr/local/lib/python3.9/ast.py”, line 105, in literal_eval
return _convert(node_or_string)
File “/usr/local/lib/python3.9/ast.py”, line 104, in _convert
return _convert_signed_num(node)
File “/usr/local/lib/python3.9/ast.py”, line 78, in _convert_signed_num
return _convert_num(node)
File “/usr/local/lib/python3.9/ast.py”, line 69, in _convert_num
_raise_malformed_node(node)
File “/usr/local/lib/python3.9/ast.py”, line 66, in _raise_malformed_node
raise ValueError(f’malformed node or string: {node!r}')
ValueError: malformed node or string: <ast.Name object at 0x7f1327022d30>
2023-08-08 04:01:07,363 - UiPath_core.trainer_run:main:98 - INFO: Job run stopped.

3

@Jack_Moran

As per error the file names contains some special characters

Loading data from /microservice/dataset/RogersTrainingData No 0X92.csv
2023-08-08 04:01:07,360 - UiPath_core.training_plugin:model_run:179 - ERROR: Training failed for pipeline type: TRAIN_ONLY, error: malformed node or string: <ast.Name object at 0x7f1327022d30>

Cheers

4

I am aware of this, but how do I figure out which special characters I should remove? There is no ASCII code like in some other error logs, and no indication of which specific string caused the problem.

5

@Jack_Moran

It mostly looks like the error in the filename itself…which contains a quote…

If that is not the case then i would suggest gow ith a smaller dataset which helps in understanding the line whichcauses the issue…mostly it might also be because of a very long string in the dataset

Cheers

6

I tried using a file with no filename (it was just called New Microsoft CSV) and only 10 rows of data… I got the same error again! Does this mean that there is a data point with a string that is too long??

7

@Jack_Moran

yes…Can you do one thing…just to get one full run…have only 3 types of data 7 each type and descriptions accrodingly…and upload for first full run then we can proceed accordingly

cheers

8

I updated the dataset to include 3 labels, with 7 examples each, and got the same error once again. What now?

9

Hello @Jack_Moran,
The error message you’re encountering, “malformed node or string: <ast.Name object at 0x7f1327022d30>”, indicates an issue related to parsing a node within the training process. Specifically, it seems to be related to some part of the code attempting to evaluate an expression using the ast module, which is the Abstract Syntax Tree module used to parse and analyze Python code.

10

I see! So what does this mean for me? How should I go about responding to this error?

11

Debug and Log

Check Documentation

Check Data Inputs

Check the code

These would be the steps I’d take.
Cheers! :slight_smile:

12

That doesn’t quite answer my question, I’m afraid. What I’m trying to figure out is how to pinpoint the cause of the issue. I don’t know of a way to look at the code AI Center is using for this model, and I don’t know how to interpret the error log’s description of the string. How can I find the string that is causing the problem?

13

@Jack_Moran

In the descriptions mention something which is easy and not big…then lets come at conclusion what is giving the issue

cheers

14

So I tried shortening every Description in the data. However, not only did the pipeline fail again, but there are almost no logs!

Train only of MultiLabelTCPack 23.5.0.0 launched - Run 7673823f-7a81-44bb-bb39-d5d3d862fce3
Train only of MultiLabelTCPack 23.5.0.0 started - Run 7673823f-7a81-44bb-bb39-d5d3d862fce3
Train only of MultiLabelTCPack 23.5.0.0 scheduled - Run 7673823f-7a81-44bb-bb39-d5d3d862fce3
Train only of MultiLabelTCPack 23.5.0.0 failed - Run 7673823f-7a81-44bb-bb39-d5d3d862fce3

Error Details : Pipeline failed due to ML Package Issue

2023-08-08 16:16:41,474 - UiPath_core.trainer_run:main:74 - INFO: Starting training job…
2023-08-08 16:16:41,474 - UiPath_core.trainer_run:main:74 - INFO: Starting training job…