Test Status

You can add a status to your tests to help manage which test needs to be worked on. The statuses are displayed as one of the columns in the Test Library list and in the Test Editor.

By default all statuses are labeled as "Draft". After the feature is enabled, Testim automatically identifies tests that were run in the last 30 days and labels them as "Active". All other statuses are applied manually, as described below.

By manually managing the tests' statuses you can enjoy the following benefits:

• Manually quarantine flaky/failing tests without taking them out of the CI/suite.
• Easily notice tests that are not connected to the CI (not active).
• Gradually add tests to the CI without failing the entire run.
• Improve project visibility by filtering tests by status.

📘

This is a PRO feature

This feature is only open to projects on our professional plan. To learn more about our professional plan, see here.

🚧

Note

In order to use this feature, you'll need to upgrade your CLI version to be at least v3.135.0, read here about CLI installation

Test Statuses

Each test can have one of the following statuses:

*The best practice is to add a test to the CI/Scheduler only after the test is ready, and not in Draft status

Viewing the test status

The tests' statuses are displayed in the following screens:
Test Library screen - Test Lists -> Tests

Test Editor

Filtering tests by status

To filter test by status:

1. Go to Test List --> Tests
2. Click the Filter button.

3. In the Filter Test pane, select the checkboxes of the relevant statuses.

Modifying the status of a test

Statuses can be modified manually. All changes to the test status will appear In the revision history.

To change the test status through the Test Library:

1. Go to Test List --> Tests
2. In the Status column, select the relevant status.

*Note: it is possible to bulk edit tests status, by selecting all the tests you would like to edit, and then clicking on change status from the top menu.

📘

As part of the test status, we also added an option to view your flaky tests and decide how to manage their status. To read more about the flaky tests, see here.

To change the test status through the Testim's Editor:

1. Open the test in the editor.
2. Change the status from the top left corner

📘

When changing the status the test is saved as a revision. If you want to merge it back to Master, you will have to resolve it as a 3 way merge conflict, to read more see here

Using the test status

On test runs

After running the tests, the statuses will be reflected in the following way:

• Draft tests that run will appear as before. It is recommended to change these tests' status to Active.
• Evaluating tests will appear in the test runs, but in a case of failure there will be an indication that the failure was ignored.

• Active tests will appear as before.
• Quarantine tests will not run (they will not appear in the test runs).

On suite runs

On suite runs, the statuses will be reflected in the following way:

• Draft tests that run will appear as before. It is recommended to change these tests' status to Active.
• Evaluating tests will appear in the test runs, but in a case of failure, there will be an indication that they did not fail the CI.
• Active tests will appear as before.
• Quarantine tests will not run (they will appear in the suite run with a quarantine indication)

CLI runs

• Evaluating failed tests will appear in the CLI run summary as FAILED-EVALUATING
• Evaluating failed tests will be added to the run XML report with a new status "failure-evaluating", to read more about the CLI XML report, see here
• Quarantine tests will be added to the run XML with a "Skipped" flag.

📘

Quarantine tests can be run in the CLI by using the bypassed “--run-quarantined-tests” flag