Hi @Jeremy_Tederry,
Thanks for your reply @Jeremy_Tederry .
I have tried to train with Full pipeline and it got failed.
Getting Logs like:
Full training of ObjectDetector 1.0 launched - Run 44bbe37f-aaf2-4c7b-a9c8-252b026401c4
Full training of ObjectDetector 1.0 scheduled - Run 44bbe37f-aaf2-4c7b-a9c8-252b026401c4
Full training of ObjectDetector 1.0 started - Run 44bbe37f-aaf2-4c7b-a9c8-252b026401c4
Full training of ObjectDetector 1.0 failed - Run 44bbe37f-aaf2-4c7b-a9c8-252b026401c4
Error Details : Pipeline failed due to ML Package Issue
yolo_weights:83 - INFO: conv2d_106
2021-10-07 06:19:46,753 - root:load_yolo_weights:87 - INFO: batch_normalization_103
2021-10-07 06:19:46,757 - root:load_yolo_weights:83 - INFO: conv2d_107
2021-10-07 06:19:46,758 - root:load_yolo_weights:87 - INFO: batch_normalization_104
2021-10-07 06:19:46,767 - root:load_yolo_weights:83 - INFO: conv2d_108
2021-10-07 06:19:46,768 - root:load_yolo_weights:87 - INFO: batch_normalization_105
2021-10-07 06:19:46,771 - root:load_yolo_weights:83 - INFO: conv2d_109
2021-10-07 06:19:46,772 - root:load_yolo_weights:87 - INFO: batch_normalization_106
2021-10-07 06:19:46,781 - root:load_yolo_weights:83 - INFO: conv2d_110
2021-10-07 06:19:46,781 - root:load_yolo_weights:87 - INFO: batch_normalization_107
2021-10-07 06:19:46,785 - root:load_yolo_weights:83 - INFO: conv2d_111
2021-10-07 06:19:46,785 - root:load_yolo_weights:87 - INFO: batch_normalization_108
2021-10-07 06:19:46,794 - root:load_yolo_weights:83 - INFO: conv2d_112
2021-10-07 06:19:46,794 - root:load_yolo_weights:87 - INFO: batch_normalization_109
2021-10-07 06:19:46,798 - root:load_yolo_weights:83 - INFO: conv2d_113
2021-10-07 06:19:46,798 - root:load_yolo_weights:87 - INFO: batch_normalization_110
2021-10-07 06:19:46,807 - root:load_yolo_weights:83 - INFO: conv2d_114
2021-10-07 06:19:46,808 - root:load_yolo_weights:87 - INFO: batch_normalization_111
2021-10-07 06:19:46,812 - root:load_yolo_weights:83 - INFO: conv2d_115
2021-10-07 06:19:46,812 - root:load_yolo_weights:87 - INFO: batch_normalization_112
2021-10-07 06:19:46,821 - root:load_yolo_weights:83 - INFO: conv2d_116
2021-10-07 06:19:46,821 - root:load_yolo_weights:87 - INFO: batch_normalization_113
2021-10-07 06:19:46,826 - root:load_yolo_weights:83 - INFO: conv2d_117
2021-10-07 06:19:46,826 - root:load_yolo_weights:87 - INFO: batch_normalization_114
2021-10-07 06:19:46,835 - root:load_yolo_weights:83 - INFO: conv2d_118
2021-10-07 06:19:46,835 - root:load_yolo_weights:87 - INFO: batch_normalization_115
2021-10-07 06:19:46,869 - root:load_yolo_weights:83 - INFO: conv2d_119
2021-10-07 06:19:46,870 - root:load_yolo_weights:87 - INFO: batch_normalization_116
2021-10-07 06:19:46,877 - root:load_yolo_weights:83 - INFO: conv2d_120
2021-10-07 06:19:46,878 - root:load_yolo_weights:87 - INFO: batch_normalization_117
2021-10-07 06:19:46,906 - root:load_yolo_weights:83 - INFO: conv2d_121
2021-10-07 06:19:46,907 - root:load_yolo_weights:87 - INFO: batch_normalization_118
2021-10-07 06:19:46,914 - root:load_yolo_weights:83 - INFO: conv2d_122
2021-10-07 06:19:46,914 - root:load_yolo_weights:87 - INFO: batch_normalization_119
2021-10-07 06:19:46,942 - root:load_yolo_weights:83 - INFO: conv2d_123
2021-10-07 06:19:46,942 - root:load_yolo_weights:87 - INFO: batch_normalization_120
2021-10-07 06:19:46,949 - root:load_yolo_weights:83 - INFO: conv2d_124
2021-10-07 06:19:46,949 - root:load_yolo_weights:87 - INFO: batch_normalization_121
2021-10-07 06:19:46,979 - root:load_yolo_weights:83 - INFO: conv2d_125
2021-10-07 06:19:46,979 - root:load_yolo_weights:87 - INFO: batch_normalization_122
2021-10-07 06:19:46,986 - root:load_yolo_weights:83 - INFO: conv2d_126
2021-10-07 06:19:46,986 - root:load_yolo_weights:87 - INFO: batch_normalization_123
2021-10-07 06:19:47,014 - root:load_yolo_weights:83 - INFO: conv2d_127
2021-10-07 06:19:47,015 - root:load_yolo_weights:87 - INFO: batch_normalization_124
2021-10-07 06:19:47,021 - root:load_yolo_weights:83 - INFO: conv2d_128
2021-10-07 06:19:47,021 - root:load_yolo_weights:87 - INFO: batch_normalization_125
2021-10-07 06:19:47,049 - root:load_yolo_weights:83 - INFO: conv2d_129
2021-10-07 06:19:47,050 - root:load_yolo_weights:87 - INFO: batch_normalization_126
2021-10-07 06:19:47,056 - root:load_yolo_weights:83 - INFO: conv2d_130
2021-10-07 06:19:47,057 - root:load_yolo_weights:87 - INFO: batch_normalization_127
2021-10-07 06:19:47,089 - root:load_yolo_weights:83 - INFO: conv2d_131
2021-10-07 06:19:47,090 - root:load_yolo_weights:87 - INFO: batch_normalization_128
2021-10-07 06:19:47,096 - root:load_yolo_weights:83 - INFO: conv2d_132
2021-10-07 06:19:47,096 - root:load_yolo_weights:87 - INFO: batch_normalization_129
2021-10-07 06:19:47,125 - root:load_yolo_weights:83 - INFO: conv2d_133
2021-10-07 06:19:47,125 - root:load_yolo_weights:87 - INFO: batch_normalization_130
2021-10-07 06:19:47,129 - root:load_yolo_weights:83 - INFO: conv2d_134
2021-10-07 06:19:47,129 - root:load_yolo_weights:87 - INFO: batch_normalization_130
2021-10-07 06:19:47,133 - root:load_yolo_weights:83 - INFO: conv2d_135
2021-10-07 06:19:47,133 - root:load_yolo_weights:87 - INFO: batch_normalization_131
2021-10-07 06:19:47,137 - root:load_yolo_weights:83 - INFO: conv2d_136
2021-10-07 06:19:47,137 - root:load_yolo_weights:87 - INFO: batch_normalization_132
2021-10-07 06:19:47,147 - root:load_yolo_weights:83 - INFO: conv2d_137
2021-10-07 06:19:47,148 - root:load_yolo_weights:87 - INFO: batch_normalization_133
2021-10-07 06:19:47,152 - root:load_yolo_weights:83 - INFO: conv2d_138
2021-10-07 06:19:47,152 - root:load_yolo_weights:87 - INFO: batch_normalization_134
2021-10-07 06:19:47,161 - root:load_yolo_weights:83 - INFO: conv2d_139
2021-10-07 06:19:47,162 - root:load_yolo_weights:87 - INFO: batch_normalization_135
2021-10-07 06:19:47,167 - root:load_yolo_weights:83 - INFO: conv2d_140
2021-10-07 06:19:47,167 - root:load_yolo_weights:87 - INFO: batch_normalization_136
2021-10-07 06:19:47,176 - root:load_yolo_weights:83 - INFO: conv2d_141
2021-10-07 06:19:47,176 - root:load_yolo_weights:87 - INFO: batch_normalization_137
2021-10-07 06:19:47,179 - root:load_yolo_weights:83 - INFO: conv2d_142
2021-10-07 06:19:47,179 - root:load_yolo_weights:87 - INFO: batch_normalization_137
2021-10-07 06:19:47,183 - root:load_yolo_weights:83 - INFO: conv2d_143
2021-10-07 06:19:47,183 - root:load_yolo_weights:87 - INFO: batch_normalization_138
2021-10-07 06:19:47,187 - root:load_yolo_weights:83 - INFO: conv2d_144
2021-10-07 06:19:47,187 - root:load_yolo_weights:87 - INFO: batch_normalization_139
2021-10-07 06:19:47,192 - root:load_yolo_weights:83 - INFO: conv2d_145
2021-10-07 06:19:47,192 - root:load_yolo_weights:87 - INFO: batch_normalization_140
2021-10-07 06:19:47,196 - root:load_yolo_weights:83 - INFO: conv2d_146
2021-10-07 06:19:47,196 - root:load_yolo_weights:87 - INFO: batch_normalization_141
2021-10-07 06:19:47,201 - root:load_yolo_weights:83 - INFO: conv2d_147
2021-10-07 06:19:47,201 - root:load_yolo_weights:87 - INFO: batch_normalization_142
2021-10-07 06:19:47,205 - root:load_yolo_weights:83 - INFO: conv2d_148
2021-10-07 06:19:47,205 - root:load_yolo_weights:87 - INFO: batch_normalization_143
2021-10-07 06:19:47,210 - root:load_yolo_weights:83 - INFO: conv2d_149
2021-10-07 06:19:47,210 - root:load_yolo_weights:87 - INFO: batch_normalization_144
2021-10-07 06:20:13,374 - uipath_core.training_plugin:trigger_full_training_and_publish_model:466 - ERROR: Failed to trigger full training and publish data, error: cannot compute Mul as input #1(zero-based) was expected to be a double tensor but is a string tensor [Op:Mul]
2021-10-07 06:20:13,375 - uipath_core.training_plugin:model_run:145 - ERROR: Training failed for pipeline type: FULL_TRAINING, error: cannot compute Mul as input #1(zero-based) was expected to be a double tensor but is a string tensor [Op:Mul]
2021-10-07 06:20:13,393 - uipath_core.trainer_run:main:81 - ERROR: Training Job failed, error: cannot compute Mul as input #1(zero-based) was expected to be a double tensor but is a string tensor [Op:Mul]
Traceback (most recent call last):
File “/home/aifabric/.local/lib/python3.6/site-packages/uipath_core/trainer_run.py”, line 76, in main
wrapper.run()
File “/microservice/training_wrapper.py”, line 57, in run
return self.training_plugin.model_run()
File “/home/aifabric/.local/lib/python3.6/site-packages/uipath_core/training_plugin.py”, line 146, in model_run
raise e
File “/home/aifabric/.local/lib/python3.6/site-packages/uipath_core/training_plugin.py”, line 134, in model_run
self.run_full_training()
File “/home/aifabric/.local/lib/python3.6/site-packages/uipath_core/training_plugin.py”, line 167, in run_full_training
self.trigger_full_training_and_publish_model()
File “/home/aifabric/.local/lib/python3.6/site-packages/uipath_core/training_plugin.py”, line 467, in trigger_full_training_and_publish_model
raise e
File “/home/aifabric/.local/lib/python3.6/site-packages/uipath_core/training_plugin.py”, line 443, in trigger_full_training_and_publish_model
self.train_model(self.training_data_directory)
File “/home/aifabric/.local/lib/python3.6/site-packages/uipath_core/training_plugin.py”, line 111, in train_model
self.model.train(directory)
File “/microservice/train.py”, line 52, in train
training.main(self.input_lr)
File “/microservice/training.py”, line 134, in main
results = train_step(image_data, target)
File “/microservice/training.py”, line 94, in train_step
lr = global_steps / warmup_steps * train_learningRate
File “/home/aifabric/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py”, line 997, in binary_op_wrapper
return func(x, y, name=name)
File “/home/aifabric/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py”, line 1283, in _mul_dispatch
return gen_math_ops.mul(x, y, name=name)
File “/home/aifabric/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py”, line 6089, in mul
_ops.raise_from_not_ok_status(e, name)
File “/home/aifabric/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 6653, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File “”, line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: cannot compute Mul as input #1(zero-based) was expected to be a double tensor but is a string tensor [Op:Mul]