Test runs

View statistics of last test runs

The "Test Runs" screen enables you to view statistics on past runs that were run by the CLI, CI, or scheduled runs. Runs performed directly from the editor are not displayed in the Test runs screen. The Test Runs screen displays aggregated statistics on all the selected tests as well as statistics on specific tests (e.g. the number of times it passed/failed and the average test duration, etc.).

How to access the Test Runs screen

There are three different ways to access the "Test Runs" screen:

  • On the main menu, click Runs and then click on the Test runs tab.
3342
  • From the Suite runs screen (Runs => Suite runs), select a run, hover over a test and click the Test history button on the right.
1140

Test Runs screen elements

The Test Runs screen displays statistics on the last runs for a specific test / all tests according to the selected filters. When you navigate to this screen from a specific test, you'll see the statistics for that test only. Clicking on a specific test run will open that test run inside the Test Editor allowing you to drill down and see the test run details e.g. screenshots and duration of each step.

1185

Aggregated Statistics

The Aggregated Statistics section displays the following summary of the results from all runs:

  • Success - the percentage of the successful runs out of all the runs that ran during the selected time.
  • Tests - the number of tests ran during the selected time
  • Passed - how many test runs succeeded
  • Avr. Duration - The average duration of the runs
3342

Filters

You can filter the statistics based on the following parameters. The filters can be aggregated by selecting multiple filters:

  • Period of time - select one of the predefined options or Custom to specify a specific start and end date.
  • Tests - select specific test or all tests. You can use the search function to narrow the list of tests.
  • Status - select a specific test run status.
  • Failure type - Failed runs are tagged with a reason. You can use this filter to display the runs that were tagged with one of the listed reasons. Selecting this filter will display failed tests only.
  • Browser - select the desired browsers that were used in the test run.
  • Result labels - select the relevant test result labels.
1130

Runs Graph

The runs graph displays the duration (y-axis) across time (x-axis). Each bar represents a test run as following:

  • Green bars - passed
  • Red bars - failed
    You can hover over a bar to view its details.
1754

Download to CSV

By default, the test runs shows up to 200 results.
When there are more than 200 results, downloading the CSV will contain all the results.

2694

Tag test failure

Click the Tag Test Failure button to tag all the failed runs in the list below with failure types . To learn more - see Tagging failed runs with failure types

594

Test Runs Details

The test runs details lists all the test runs that match the conditions of the filters.

ParameterDescriptionValues
Test NameThe name of the test in the test run.
BrowserThe browser that was used in the test run.
StartedThe time the run has started.
DurationThe duration of the test run.
Result LablesThe result label that was added to the run. Click to view the labels.
Failure TagThe failure tag that was added to the run. To add a Failure Tag, click the "Tag test failure" link. Lear more - Tagging failed runs with failure types
StatusThe test run status. Failed - red x
Failed with retries - red x with yellow exclamation mark
* Passed - green v

Viewing retries

When a test run had a few retries, it is possible to view the retries' results. Test run with retries will have a yellow (!) marking on the status icon.
To read more about retries and how to set them up, see here

3198

To view all retry results for a specific run:

  1. Hover the result line
  2. Click the "Test retries" icon
  3. Open the result you would like to view
1216

📘

In case your plan does not support viewing retries, please contact support.

2600