Testim Documentation

Welcome to the Testim documentation. You'll find comprehensive guides to help you get started quickly, example code snippets, and tips for being more productive. You will also find details about advanced features that allow you to customize Testim to fit your environment or special testing conditions. Don't worry, we'll support you if you get stuck. Let's jump right in!

Guides    Changelog

Data-driven testing

Learn how to run the same test with different data

Data-driven testing lets you define parameters in your tests and use them in any step. You can define multiple ordered data sets and have the test run against each set separately using the Testim CLI.

πŸ“˜

When using data sets, previous data will not be automatically removed from test until explicitly removed.

Adding test data to your tests

Test data can be added to a test in three ways:

  • Through the visual editor (UI) - test data is added through the Test Data property in the Setup step. A Data Set is defined in JavaScript and multiple ordered Data Sets are defined by a JS Array literal of objects. See Configuring a Data Driven Test from the visual editor.
  • Through the Config file - the configuration file is a common JS containing all the required parameters to run your test suite and run hooks. Through one of these hooks it is possible to add test data to specified tests. This data set can override the dataset that was defined in the visual editor. See Configuring a Data Driven Test from the visual editor. The test can be run using the CLI, while adding a flag to use the config file in this run.
  • Through an external file- Test data from external sources (e.g. CSV, DB etc.) can be passed to one or multiple tests using the Config file. The test can be run using the CLI, while adding a flag to use the config file in this run.

Popular use cases for Data Driven Testing

  • Run a signup test with different data - when you want to test your signup page with different user name and password, you can use Test Data and Conditions to build one test in Testim that will run several times with different usernames and passwords (negative or positive)
  • Read\inject data from a database - when you want to inject data into a database and then pass the data that was generated by the test. To generate the data you can use Testim CLI hooks and after the injection of the data was successfully completed, you can pass the data to your test.

Configuring a Data Driven Test from the visual editor

Test Data can be added to a specific test in the UI by choosing "Test Data" in the setup step's properties panel
To add Test Data to a test:

  1. In the test's Setup step (the first step), click the Show properties button ().
  2. Click on Test Data.
  3. Define the data set in the JS editor
    This is the data set we defined in the example:
return [{
  "username": "tomsmith",
  "password": "SuperSecretPassword!"
},{
  "username": "david",
  "password": "SecretPassword?"
}];

πŸ“˜

The brackets must remain in the return line.

πŸ“˜

Only when you run your test from Testim CLI, scheduler or local suite run, the test will run multiple times, in order, each time with a different Data Set.

  1. Click the Show Properties button () on the the step(s) that you want to add the data set to. For example, in the "set username" and "set password" steps.
  2. In the Text to assign field, replace the existing text with the name of the parameter. For example, in the Set username step, enter the username parameter.
  1. Run your test.
    The test will run with the first Data Set:

"username": "tomsmith", "password": "SuperSecretPassword!"

Running the test from the editor will only run the first data set. When you want to run additional data sets, you will need to use the CLI, which includes the beforeSuite hook that overrides test data provided in the UI, or the Scheduler.

Adding test data by uploading a CSV/Excel file

It is also possible to upload the test data using a CSV or Excel file. After the file is uploaded its data will be added as the test data according to the following structure:

  • First row - parameter name
  • Consequent rows - includes the data set.

πŸ“˜

As part of this method the uploaded data will not be updated if the file has changed. Every update will require uploading the file again. In contrast, when using the config file method (see, Data Driven tests using data from an external source), the file will be automatically parsed on every run.

To add test data by uploading a file:

  1. Prepare an Excel/CSV file according to the structure mentioned above.
  1. In the test's Setup step (the first step), click the Show properties button ().
  2. Click on Test Data.
  3. Click Upload File and select the file you have created.
  1. The uploaded data is displayed. You can modify this data if needed.
  1. Use the parameters in the test steps as explained above.

πŸ“˜

Only when you run your test from Testim CLI, scheduler or local suite run, the test will run multiple times, in order, each time with a different Data Set.

Note - when updating the test data in the editor, it is revisioned.
This means 2 things:

  • You can revert back to previously saved test data through the revisions history
  • When changing the test data in a branch, while merging the user will need to decide if they want to apply the change on the merge as well

Configuring Data Driven Tests using the Config file

Test Data can be passed to one or multiple tests by editing the config file and then running the test with the CLI, while adding a flag to use the config file in this run.

πŸ“˜

The Config file code is processed by the CLI that runs the test on the Testim grid, so it is possible to run JS that is supported in Node and not on the browser.

To add test data to a config file:

  1. Create a config file or edit an existing one.
  2. Add tests names and data sets (data sets should be in JSON format), by adding them to the overrideTestData under the beforeSuite hook. All other parts of the config file should remain/be included.
    Here is an example of a beforeSuite hook in the configuration file :
beforeSuite: function () {
    return {
       overrideTestData: {                 
          "Test 1": {user: "dave", password : "123"},
          "Test 2": {name: "ryan"}                    
        }
    }    
} //add comma here if there are more functions after beforeSuite

Here is the same example with 2 datasets for the first test :

beforeSuite: function () {
    return {
       overrideTestData: {                 
          "Test 1": [{user: "michelle", password : "belle"},
                     {user: "paul", password : "walrus"}]
          "Test 2": {name: "john"}                    
        }
    }    
} //add comma here if there are more functions after beforeSuite

If the one of the data set parameters is missing from a consequent data set, for example, if the second data set did not include the password (i.e. did not include password: walrus), Testim will use the parameter data from the first data set (i.e. belle).

To run a test using the config file:
After creating the config file, pass the config file to the Command Line (CLI) runner as a parameter, while making sure to indicate the path to the file if needed. When you run your test from Testim CLI, the test will run multiple times, in order, each time with a different Data Set.

testim -c "testimConfig.js"

If the test already included data (e.g. that was specified in the visual editor), this data will override the original test data, if the Config file includes the override function naming the specific test.

Data Driven tests using data from an external source

Test data from external sources (e.g CSV, DB etc.) can be passed to one or multiple tests using the config file and then running the test using the CLI, while adding a flag to use the config file in this run.

To add external test data to a config file:

  1. Create a config file or edit an existing one.
  2. Install the npm package csvtojson in order to use this functionality. See more details here: https://www.npmjs.com/package/csvtojson. The csvtojson npm package should be included in the Config file as shown below (const csvtojson = require("csvtojson")
  3. Add a JavaScript function that loads data from an external source into JSON objects in the beginning of the config file (loadCsvFile(path)).
// JS function that loads data from CSV into Json objects
const csvtojson = require("csvtojson");
function loadCsvFile(path) {
    return new Promise((resolve) => {
        return csvtojson()
            .fromFile(path)
            .then(resolve, err => {
                console.error("failed to read csv file", err.message);
                resolve([]);
            });
    });
}
  1. Use overrideTestData in the beforeSuite to pass test names and their data sets as shown below.
beforeSuite: function () {
    return Promise.all([loadCsvFile('./data.csv'),
    loadCsvFile('./data2.csv'), loadCsvFile('./data3.csv')])
        .then(([jsonObj, jsonObj2, jsonObj3]) => {
            return {
                BEFORE_SUITE: "BEFORE_SUITE",
                overrideTestData: {
                "Test 3": jsonObj,
                "Test 4": jsonObj2,                        
                "Test 5": jsonObj3,                         
                // You can also pass static datasets to tests 
                "Test 6": {name: "ryan"}                    
            }
        }
    });
} //add comma here if there are more functions after beforeSuite

πŸ“˜

Test data provided in the beforeSuite() hook in the config file overrides test data provided in the UI.

πŸ“˜

In the CSV file the first row (header row) will contain the data keys and will be used as parameter names, and each row afterward will contain a dataset of values.

To run a test using the config file:
After creating the config file, pass the config file to the Command Line (CLI) runner as a parameter, while making sure to indicate the path to the file if needed. When you run your test from Testim CLI, the test will run multiple times, in order, each time with a different Data Set.

testim -c "testimConfig.js"

Viewing test runs with different data sets

  1. Go to Suite runs view and go to the suite that you just run.
  2. You can see the run result with the Data Set per test when you hover the "i" icon in the right side of the test and then click the See Data Set button.

Setting Cookies

If you need cookies to be loaded before your system loads or you need cookies for a particular test, You can also define cookies using the reserved cookies parameter. The cookies parameter can be added through the visual editor and through the Config file.

Read all about the setting cookies options here.

Updated 14 days ago


Data-driven testing


Learn how to run the same test with different data

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.