Testim.ioChangelogStart Testim Free

Why did my test fail?

Understand error messages, why they occur, and how to fix them.

Suggest Edits

After running a test, you may find that the test fails and Testim displays an error message with some details about the failure. In general, tests fail for two main reasons:

  • A defect was identified in the tested application and the test did not return the expected results.
  • There was a flaw in the way the test was created.

Testim identifies several different failure types:

  1. Element not visible: The element exists on the page but is not visible.
  2. Element not found: Testim couldn't detect the element on the page.
  3. Tab not found: Testim could not find the correct tab for the step to act on.
  4. Frame not found: Testim could not find the correct frame for the step to act on.
  5. JavaScript Error: An error occurred while evaluating a JavaScript step or expression.
  6. Could not get browser: There is no browser available in the grid, all the browsers in the grid are in use.
  7. Browser type is not supported: The browser you requested to run on is not supported on your grid or the mode does not match the browser.
  8. Page is not available: The application is not available from the location from which it is running.
  9. Failed to set text: Setting text in the input text resulted in an empty field.
  10. Failed to click: The click step passed, however, the click wasn't recognized by the app being tested.
  11. Concurrency limit reached: Too many tests were run concurrently.
  12. Test Timed Out: The test took too long to run.
  13. API Step Failed: An API step did not provide the expected response.

Test Failure Types

When a test fails, Testim will identify which step of the test failed and display an error message related to the type of failure.

1. Element not visible

This error occurs when the target element exists on the page but is not visible.

Common Causes:

  • The target element is currently not in the viewport.
  • The target element visibility is hidden or set to be: "display: none".
  • The size (height or width) of one element in your test equals to 0.
  • There is an overlay with a high z-index which is placed above the element.
  • The opacity of an object is set to 0.

**What can I do?**

  • Look at the side-by-side screenshots to determine if the element was visible in the test.
  • Update CSS in your application to ensure the element is not hidden, has correct opacity, and has the expected the size and location.
  • Check if the resolution of the recording is the same as the computer running the test. If not, adjust the resolution on the computer running the test to match the original recording.
  • If the element does not appear in the viewport, you may need to add a scroll step to reach it.
  • Sometimes elements only become visible when the mouse hovers over another element (e.g. a drop-down menu). As hover steps are not recorded automatically, you may need to record one to make your element visible. See Hover Step for more details on how to do this.
  • Turn off the "element must be visible" checkbox in the step's properties. Sometimes you will find that the visibility validation is not required for your business logic, in which case you can turn it off. If there are overlays in the system, it can be a quick solution. See Editing Target Element Properties for more details.
  • Check the failed step DOM: When a step fails, Testim saves a complete DOM snapshot, so you can later debug. For more details, see Debug by seeing the failed step DOM.

2. Element not Found

This error occurs when Testim couldn't detect the element on the page.

Common Causes:

  • The element has been removed from the page.
  • The element name or ID has been changed.
  • The previous steps passed but the test did not progress to the screen where the target element was expected. An example of this is when the test is expecting a username and password entry to login, but the test’s recorded login information is incorrect or has changed.
  • The page didn’t finish loading or took too long to load.

What can I do?

  • Compare the side-by-side screenshots to determine if the element was visible in the test.
  • View the screenshot progression to determine if the previous step failed to navigate to the page.
  • Ensure all form submission information is correct. See Parameters for more info.
  • If the page didn't finish loading or has a delay in loading, add a wait step to force Testim to wait for the target element to be visible.
  • If you can see the element but it is not recognized, try Improving the target element.
  • Check the failed step DOM: When a step fails, Testim saves a complete DOM snapshot, so you can later debug. For more details, see Debug by seeing the failed step DOM.

3. Tab not Found

This error occurs when Testim could not find the correct tab for the step to act on.

Common Causes:

  • Test opened a new browser window instead of a new tab.
  • Opening of new tab was blocked by pop-up blocker.

What can I do?

  • When running the test, if you see a red warning near your browser's navigation bar, click it and allow pop-ups for that domain.
  • Update your browser's advanced settings and pop-up settings to always allow new browser tabs and windows for that domain.

4. Frame not Found

This error occurs when Testim could not find the correct frame for the step to act on.

Common Causes:

  • Frame did not load.
  • Expected iframe no longer exists.

What can I do?

  • Compare the side-by-side screenshots to determine if the frame was visible in the test.
  • Inspect the DOM to validate the expected frame was loaded.
  • If target element is no longer in an iframe, reassign the target element for the failed test step.

5. JavaScript Error

This error occurs when JavaScript step or expression is not correctly evaluated.

Common Causes:

  • “Syntax error” - Incorrect JavaScript syntax. The error can be due to code in a custom step or code in a JavaScript parameter.
  • "Assertion failed" - A custom validation step returned false.

What can I do?

6. Could not get Browser

This error occurs when the expected browser was not available in the grid when running the test.

Common Causes:

  • No browser is available in the grid
  • Al browsers in the grid are in use.
  • Browser used in the text is not available in your grid.

What can I do?

  • Check if your grid contains the browser you tried to run on.
  • If you are using the parallel parameter in the CLI, ensure you are not running tests in parallel that use all available browsers on your grid.
  • Add more browsers to your grid.
  • If all tests fail with this error, there might be a problem with the grid settings. Contact your grid provider for additional help.

See Grid Management for details on your current grid configuration.

7. Browser Type is not Supported

This error occurs when a test is run on an unsupported browser.

Common Causes:

  • The test was run on a browser not supported by your grid.
  • The test was run in a mode that does not match the browser.

What can I do?

  • Make sure the browser exists on your grid. If it doesn't, you can contact your grid provider for additional help.
  • IE, Edge, Firefox and Safari browsers need to run in selenium mode, make sure your CLI contains "--mode selenium".

8. Page is not Available

The application is not available from the location from which it is running.

Common Causes:

  • The base URL is incorrect.
  • Application is not available from the grid.

What can I do?

  • Confirm the Base URL for the test is correct.
  • If you're running from a remote grid, confirm your application is available from the grid. If it isn't, you can contact us to enable a tunnel.

9. Failed to Set Text

This error occurs when a Set Text step resulted in an empty field.

Common Causes:

  • Target element could not be found.
  • Input field does not accept the format of the text assigned to the step. For example, the step is trying to input an email address into a form field that only allows alpha-numeric characters.

What can I do?

  • Compare the side-by-side screenshots to confirm the target input field is present on the page.
  • Confirm the target element allows the expected input.
  • Update the Set Text step properties to allow “Native Events.”

10. Failed to Click

This error occurs when a Click Step passed but the click wasn't actually executed.

Common Causes:

  • Testim click event not recognized by the application.
  • Page was not fully loaded while the click step was already executed (timing issue).

What can I do?

  • Run the test manually to confirm the target element can be clicked.
  • Add a short "sleep step" or "wait for step" before the "click step" to ensure the page is fully loaded and ready for the click/set text event to be triggered.
  • Update the Set Text/click step properties to allow/remove “Native Events”.

11. Concurrency Limit Reached

This occurs when too many tests are running concurrently and the grid does not support all running tests.

Common Causes:

  • Your grid(s) do not support the number of concurrent tests you are running.

What can I do?

  • Increase the number of grids available for testing (See Grid management).
  • Reduce the number of tests running in parallel.

12. Test is Too Long (Time Out)

If a test step takes too long to complete, the test will timeout and result in a failure.

Common Causes:

  • Test contains too many steps.

What can I do?

  • Reduce the number of steps in your test.

13. API Step Failed

If an API step fails, the incorrect results of the API call may prevent the test from completing successfully.

Common Causes:

  • API endpoint URL is incorrect.
  • API response did not provide expected results.

What can I do?

  • Check the endpoint URL manually.
  • Run additional code on the API call and display a validation error to help identify the issue. (see API testing)
  • Review API Parameters are correct.

Common Troubleshooting Steps

There are two main troubleshooting steps you can take to help identify the cause of a test failure:

  • Compare screenshots: by comparing the screenshots of the original test that passed and the most recent test that failed, you can identify changes in the UI that might have caused the failure.
  • Check test logs: by investigating the console and network logs you can get insight into errors, warnings, and other info at the console level.
  • Check the failed step DOM: When a step fails, Testim saves a complete DOM snapshot, so you can later debug. For more details, see Debug by seeing the failed step DOM.

Compare Side-by-Side Screenshots

In most cases, you can identify the cause of a test failure by comparing the screenshots from the baseline test with the failed test run.

To compare side-by-side screenshots of a failed test:

  1. Navigate to Runs > Test Runs and open the failed test.
  2. Select the View Screenshot button on the test step that failed.
  3. Testim displays a side-by-side comparison of the baseline and test results screenshot. Review the Baseline screenshot, which was taken when the test was last recorded, and compare it with the Result screenshot, which was taken when the test was last ran and failed.

📘

Note:

Testim highlights the target element that caused the error. In the example above, the “Hello, John” link in the top right of the screen was present in the baseline recording but missing in the recent test run.

Check Test Logs

As a test runs, Testim captures console and network level information. See Advanced Debugging Options for more details. After a test finishes, you can review the Console Log and Network Log to get more details about any errors in your test.

  • Console Log: displays error messages and other console level information such as warnings and info.
  • Network Log: tracks network connections and when resources are being downloaded or uploaded. This log is used when you need to make sure that resources are being transferred as expected.

To review the Console Log:

  1. Navigate to Runs > Test Runs and open the failed test.
  2. Hover over the … icon in the top right corner of the test and select View console log.

To review the Network Log:

  1. Navigate to Runs > Test Runs and open the failed test.
  2. Hover over the … icon in the top right corner of the test and select View network log.

Updated 19 days ago