The "Test Runs" screen enables you to view statistics on past runs that were run by the CLI, CI, or scheduled runs. Runs performed directly from the editor are not displayed in the Test runs screen. The Test Runs screen displays aggregated statistics on all the selected tests as well as statistics on specific tests (e.g. the number of times it passed/failed and the average test duration, etc.).
There are three different ways to access the "Test Runs" screen:
- On the main menu, click Runs and then click on the Test runs tab.
- From the Suite runs screen (Runs => Suite runs), select a run, hover over a test and click the Test history button on the right.
The Test Runs screen displays statistics on the last runs for a specific test / all tests according to the selected filters. When you navigate to this screen from a specific test, you'll see the statistics for that test only. Clicking on a specific test run will open that test run inside the Test Editor allowing you to drill down and see the test run details e.g. screenshots and duration of each step.
The Aggregated Statistics section displays the following summary of the results from all runs:
- Success - the percentage of the successful runs out of all the runs that ran during the selected time.
- Tests - the number of tests ran during the selected time
- Passed - how many test runs succeeded
- Avr. Duration - The average duration of the runs
You can filter the statistics based on the following parameters. The filters can be aggregated by selecting multiple filters:
- Period of time - select one of the predefined options or Custom to specify a specific start and end date.
- Tests - select specific test or all tests. You can use the search function to narrow the list of tests.
- Status - select a specific test run status.
- Failure type - Failed runs are tagged with a reason. You can use this filter to display the runs that were tagged with one of the listed reasons. Selecting this filter will display failed tests only.
- Browser - select the desired browsers that were used in the test run.
- Result labels - select the relevant test result labels.
The runs graph displays the duration (y-axis) across time (x-axis). Each bar represents a test run as following:
- Green bars - passed
- Red bars - failed
You can hover over a bar to view its details.
By default, the test runs shows up to 200 results.
When there are more than 200 results, downloading the CSV will contain all the results.
Click the Tag Test Failure button to tag all the failed runs in the list below with failure types . To learn more - see Tagging failed runs with failure types
The test runs details lists all the test runs that match the conditions of the filters.
|Test Name||The name of the test in the test run.|
|Browser||The browser that was used in the test run.|
|Started||The time the run has started.|
|Duration||The duration of the test run.|
|Result Lables||The result label that was added to the run. Click to view the labels.|
|Failure Tag||The failure tag that was added to the run. To add a Failure Tag, click the "Tag test failure" link. Lear more - Tagging failed runs with failure types|
|Status||The test run status.|| Failed - red x|
Failed with retries - red x with yellow exclamation mark
* Passed - green v
When a test run had a few retries, it is possible to view the retries' results. Test run with retries will have a yellow (!) marking on the status icon.
To read more about retries and how to set them up, see here
To view all retry results for a specific run:
- Hover the result line
- Click the "Test retries" icon
- Open the result you would like to view
In case your plan does not support viewing retries, please contact support.
Updated about 1 year ago