Recently I have started working on Test Automation. Test automation project comes with a default frame work of “Given-when-Then”. This framework is pretty linear. It does not have a retry mechanism. We need a retry mechanism for test suite projects. Right now, if some test case fails with an application exception, script will stop at that point. Now it could be because of a genuine product issue, or script issue. We do not want to bring this failure to the notice of product team if it is a script issue.
We can overcome this with retry mechanism. We should have an option within the scheduler to set the value for retry if a test case fails*(Like we set retry for queues)*.
Having retry mechanism for test cases has the following advantages.
- If all retry attempts failed, then the failure is probably consistent and reproducible. I would expect it to be a clear functional failure that would be fast and easy to report. I jump on these first to get them out of the way.
- If some retry attempts passed, then the failure is intermittent, and it will probably take more time to investigate. I will look more closely at the logs and screenshots to determine what went wrong. I will try to exercise the product behavior manually to see if the product itself is inconsistent. I will also review the automation code to make sure there are no unhandled race conditions. I might even need to rerun the test multiple times to measure a more accurate failure rate.