Testim Documentation

Welcome to the Testim documentation. You'll find comprehensive guides to help you get started quickly, example code snippets, and tips for being more productive. You will also find details about advanced features that allow you to customize Testim to fit your environment or special testing conditions. Don't worry, we'll support you if you get stuck. Let's jump right in!

Guides    Changelog

Command line interface (CLI)

Run all your tests from the command line and integrate with your CI

Prerequisites

Node.js version >= 10.0.0

CLI Installation

Install Testim package.

npm install -g @testim/testim-cli

That’s it!

Basic CLI

You can find the basic CLI of your project on the settings page.
This CLI already includes the parameters of your token and project ID.
Choose the grid you want to run on from the grid list.
Copy the CLI, edit and add parameters according to your needs.
You can run a specific test, label, configuration, etc.
Use the list below to find the different parameters you can use in your CLI.

The common parameters

Here is an example of running the command with the most common options (see below detailed description for all the available options):

--project: which project
--token: so we'll know it's you
--label: what tests to run
--grid: name of the grid to run the tests on
--report-file: where to put the results (so CI server could read)

testim --label "<YOUR LABEL>" --token "<YOUR ACCESS TOKEN>" --project "<YOUR PROJECT ID>" --grid "<Your grid name>" --report-file test-results/testim-tests-report.xml

All CLI Parameters

Project:

--project Select which project-id to run tests from.

Tip! you can find the project-id in the IDE under settings

--project AOL-12323-a4b2-4762-df380

Access token

--token Provide your access token you got from us. If you need one please contact us: [email protected]

--token aaaaa-1234-1234-bbbb-1234qwer1234qwer

Label

--label or -l Run all tests comprising one of the mentioned labels.

Note that you can also run more than one label, by adding more --label arguments

-l sanity -l custom-label2

Test Name

--name or -n, specify test name to run.

Note that you can also run more than one test by name, by adding more --name arguments

--name "login_to_app"

Test id

--testId or -t, specify test id to run.

Note that you can also run more than one test by id, by adding more --testId arguments

--testId "5e1c62b2-7c30-635b-6441-dd804126118b"

Grid Name

--grid to specify the Selenium Grid name to use. You can use "Testim-Grid"/ your local selenium grid/ Saucelabs / Browserstack

--grid
Note: Read here how to configure your grid.

Report File

--report-file or -r specify where to print the report (by default to the output stream). The file is in the format of JUnitXMLReporter. This is used to integrate Testim results with the CI display. Usually, you'll need to set the build config in the CI to look for that file so make sure the CLI param value and the build config are set to the same location.

-r ~/report.xml

To override the classname in the report file, add this parameter :

--override-report-file-classname com.someName

Base URL

--base-url Starting URL after browser opens

--base-url staging.example.com

Test Config

test-config Specifies a configuration name (s) (browser, operating system, resolution) that will override the configuration defined for all tests in this run.
Read more here.

--test-config "1280x1024_SXGA_chrome" --test-config "1366x768_WXGA_firefox"

Mode selenium

When you run IE11, Edge, Firefox or Safari, you'll need to add this parameter before you can run.

--mode selenium

Test Suite

Suite option specifies test suite name to run. You can run more than one suite.

--suite "suite_name" --suite "suite_name2"

Test Plan

Test plan option specifies test plan name to run. You can run more than one plan. Since you define in the test plan the grid, test list, configuration, etc., you cannot override it from the CLI. Therefore when using --test-plan you cannot use these flags: --testId, --label, --name, --browser, --test-config, --test-config-id or --suite.

--test-plan "Test Plan Demo"

Execution Name

The name of the execution in Testim suite runs.

--override-execution-name "Dev Sanity"

Class Name

The class name in the JUnitXML report file is Testim.io by default. To have the suite name in the class name instead, add the parameter below with no value. If you are looking to override the class name with a specific name, add it to the parameter.

--override-report-file-classname
--override-report-file-classname "Regression"

Result Labels

The result Labels option allows you to add textual labels to your remote runs. These labels will be shown in the runs page.

--result-label "V1.40"

Test timeout

Timeout time to abort the test run if a timeout has elapsed. The timeout should be in milliseconds. The default is set to 10 minutes.

--timeout 120000

Branches

To run on a specific branch:

--branch <branch-name>

Parallel
How many tests should run in parallel

--parallel <parallel-count>

Config file

--config-file or -c Specify all the configuration options from external configuration File.
Place the config file in a path accessible by the shell, specify in the command the path to the file.

--config-file testimConfig.js

Read more about configuration file here.

Failed test retries

When this flag is used, a failed test will be executed repeatedly until either the test passes or the max number of retries has been reached - in which case the test will fail.

When a test passes after one or more retries, it will be indicated in the UI as follows:

--retries <max_num_of_retries>

Disable TestRail reporting

To suppress sending suite run results to TestRail.

--suppress-tms-reporting

Dedicated Run Tunnel

Tunnels let you run your app from an internal server/localhost and view it in an external browser.
--tunnel #default application port 80

testim --tunnel --tunnel-port <APP PORT default 80>

For more info check out: This Article

Rerun failed tests

The rerun failed tests option allows you to rerun all test which failed in a suite run.
Copy the suite run id from the suite's page :

Add the --rerun-failed-by-run-id flag with the suite's run id to the CLI :

--rerun-failed-by-run-id <run-id>

Set data retention

When you add the parameter --set-retention [whole number between 1 to 10] to a CLI run, all the test results of this execution will be marked for deletion (retention period) after the number of days specified in the parameter

Note: Our plan includes 30 days retention by default, for longer retention time, please reach out to support.

--set-retention <number-between-1-10>

Abort CLI run

It is possible to abort a CLI run, by using the CTRL+c shortcut in your terminal. The run will have "Aborted" status under the run-list

The CLI can also be forced to quit by pressing CTRL+C multiple times or closing the terminal window.
This will cause the run to stop executing but the status will remain "Running" in the editor. The run status will change to “Timeout” after 120 minutes.

Updated 5 days ago

Command line interface (CLI)


Run all your tests from the command line and integrate with your CI

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.