Test Configuration

REQUIRES ENTERPRISE EDITION

The configuration is done using a JSON object. Please note the text must use double quotes (") around all strings. The following parameters can be set:

  • xtcType: this should always be "XTC", unless a custom project template was created by Matrix Requirements staff
  • cloneSources: this should always be ["TC", "UC"], unless a custom project template was created by Matrix Requirements staff
  • presetFields: if set the "create test forms" dialog will ask for default values of these to set them in created forms. The available options depend on the project setup. The default are version, tester and executionDate
  • UC, TC: changes the columns of use cases and test cases. Note when converting TC or UC into XTC, only columns will be copied where the field of the column is identical in the TC/UC and the XTC (the editor can be different, e.g. none in XTC)
    • columns editors can be
      • none: readonly
      • text: plain text
      • textline: simple text
      • design: references to other design items
      • uprules: references from design items which can trace to the item, These are also added as traces when saving
      • downrules: references to design items to the item can trace, These are also added as traces when saving
      • rules:categ1,categ2,categ3: references to items specified by comma separated list of acceptable categories
      • result: per step pass fail field (see below)
      • user: as user
      • versionletter: a drop down with a letter
      • date: date picker
      • select:rendered as dropdowns. The select options are defined in an additional parameter "options":{"val0":"text 0", "val1":"text 1"}
  • XTC: changes the columns of executed test cases.
  • defaultTestResultResult: this is the initial value a an executed test case should have when created. Must be a 'code' from the values in 'automatic' or 'manual' section below.
  • automatic: if the overall test result should be generated from the individual test step results, this can be configured here. It is an array of items defining rules. The first rule which applies determines the status of the test result.
    • human: a human readable text explaining the test result
    • code: a unique identifier for the status of the test result
    • render: whether test result means 'ok' (all good), 'error' (test failed), 'warning' (e.g. not completed): this is used in report renderings
    • rule: the type of rule to determine from the test steps the overall status. Can be 'one', 'all' or ''. The first rule matching determines the status
    • param: parameter for the rule, i.e. a code of a test step
  • manual: These allow the user to set the overall test result manually.
    • human: a human readable text used in reports explaining the test result
    • code: a unique identifier for the status of the test result
    • render: whether test result means 'ok' (all good), 'error' (test failed), 'warning' (e.g. not completed): this is used in report renderings
    • command: the string in the drop down menu
  • perStep: Possible test results per test step.
    • human: a human readable text used in reports explaining the test result
    • code: a unique identifier for the status of the test result
    • render: whether test result means 'ok' (all good), 'error' (test failed), 'warning' (e.g. not completed): this is used in report renderings
    • command: the string in the drop down menu
    • key: the keyboard shortcut
    • image: so far only internally used to show icons for common states (passed, fail and docu)


Example Configuration
{
    "xtcType": "XTC",
    "cloneSources": ["TC", "UC"],
    "presetFields":[ 
	  {"field":"name", "value":"Version"},
      {"field":"name", "value":"Tester" } ],
    "render": {
      "UC": {
       "columns":[
         {"name": "Action UC", "field": "action", "editor": "text"  },
         {"name": "Expected Result UC", "field": "expected", "editor": "text"  }]
      },
      "TC":{
        "columns":[
          {"name": "Action 1 TC", "field": "action","editor": "text"  },
          {"name": "Action 2 TC", "field": "action2", "editor": "text" },
          {"name": "Expected Result", "field": "expected", "editor": "text"  }]
      },
      "XTC":{ 
        "columns":[
          {"name": "Action", "field": "action","editor": "none" },
          {"name": "Action Alt","field": "action2","editor": "text" },
          {"name": "Expected","field": "expected","editor": "none" },
          {"name": "Passed/Failed","field": "result","editor": "result" },
          {"name": "Comment","field": "comment","editor": "text"}]
      }
    },
    "defaultTestResultResult":"an",
    "automatic": [
        {"human": "passed", "code": "ap", "render": "ok", "rule": "all", "param": "p"},
        {"human": "failed", "code": "af", "render": "error", "rule": "one", "param": "f"},
        {"human": "not started", "code": "an", "render": "warning", "rule": "all", "param": ""},
        {"human": "documentation", "code": "ad", "render": "warning", "rule": "one", "param": "d"},
        {"human": "in progress", "code": "ai", "render": "warning", "rule": "", "param": ""}
    ],
    "manual": [
        {"human": "passed", "command": "passed", "render": "ok", "code": "p"},
        {"human": "failed", "command": "failed", "render": "error", "code": "f"},
        {"human": "", "command": "not executed", "render": "warning", "code": "r"},
        {"human": "to be decided", "command": "to be decided", "render": "warning", "code": "tbd"},
        {"human": "documentation", "command": "documentation", "render": "warning", "code": "d"},
        {"human": "in progress", "command": "in progress", "render": "warning", "code": "i"}
    ],
    "perStep": [
        {"human": "passed", "command": "passed", "render": "ok", "code": "p", "key": "p", "image": "success.png"},
        {"human": "failed", "command": "failed", "render": "error", "code": "f", "key": "f", "image": "fail.png"},
        {"human": "documentation", "command": "documentation", "render": "warning", "code": "d", "key": "d", "image": "docu.png"},
        {"human": "", "command": "not executed", "render": "warning", "code": "", "key": "r", "image": ""}
    ]
}


This example

  • adds a column to TC where users can select and design item
  • another column to XTC where user can select a FILE item

example with refs as columns
// 1) in the traceability of XTC add downrule, e.g. to a NEW category FILE (which you need to create....)
{
  "message": "files", "name": "can have file", "rule":"can_have", "any_of": [ "FILE" ] 
} 
// modify the xtc_config
{
  "xtcType": "XTC",
  "cloneSources": ["TC", "UC"],
  "presetFields":[  {    "field":"meaning",     "value":"Version"   },   {      "field":"name",     "value":"Tester"     } ],
  "render": {
    "UC": {
      "columns":[    
         {"name": "Action",          "field": "action",          "editor": "text"          },
         {"name": "Expected",          "field": "expected",          "editor": "text"          }]    },
    "TC":{
      "columns":[
        {
          "name": "Links",
          "field": "links",
          "editor": "design"  
        },
        {
          "name": "Action",
          "field": "action",
          "editor": "text" 
        },
        {
          "name": "Expected",
          "field": "expected",
          "editor": "text"  
        }]
    },
    "XTC":{
      "columns":[
        {
          "name": " Links",
          "field": "dref",
          "editor": "none" 
        },
        {
          "name": "Action",
          "field": "action",
          "editor": "none" 
        },
        {
          "name": "Expected",
          "field": "expected",
          "editor": "none" 
        },
        {
          "name": "Passed/Failed",
          "field": "result",
          "editor": "result" 
        },
        {
          "name": "Test Results",
          "field": "files",
          "editor": "downrules"  
        },
        {
          "name": "Comment",
          "field": "comment",
          "editor": "text"
        }]
    }
  },
  "defaultTestResultResult":"an|warning|not started",
  "automatic": [
    {
      "human": "passed",
      "code": "ap",
      "render": "ok",
      "rule": "all",
      "param": "p"
    },
    {
      "human": "failed",
      "code": "af",
      "render": "error",
      "rule": "one",
      "param": "f"
    },
    {
      "human": "not started",
      "code": "an",
      "render": "warning",
      "rule": "all",
      "param": ""
    },
    {
      "human": "documentation",
      "code": "ad",
      "render": "warning",
      "rule": "one",
      "param": "d"
    },
    {
      "human": "in progress",
      "code": "ai",
      "render": "warning",
      "rule": "",
      "param": ""
    }
  ],
  "manual": [
    {
      "human": "passed",
      "command": "passed",
      "render": "ok",
      "code": "p"
    },
    {
      "human": "passed with deviation",
      "command": "passed with deviation",
      "render": "ok",
      "code": "pf"
    },
    {
      "human": "failed",
      "command": "failed",
      "render": "error",
      "code": "f"
    },
    {
      "human": "",
      "command": "not executed",
      "render": "warning",
      "code": "r"
    },
    {
      "human": "to be decided",
      "command": "to be decided",
      "render": "warning",
      "code": "tbd"
    },
    {
      "human": "documentation",
      "command": "documentation",
      "render": "warning",
      "code": "d"
    },
    {
      "human": "in progress",
      "command": "in progress",
      "render": "warning",
      "code": "i"
    }
  ],
  "perStep": [
    {
      "human": "passed",
      "command": "passed",
      "render": "ok",
      "code": "p",
      "key": "p",
      "image": "success.png"
    },
    {
      "human": "failed",
      "command": "failed",
      "render": "error",
      "code": "f",
      "key": "f",
      "image": "fail.png"
    },
    {
      "human": "documentation",
      "command": "documentation",
      "render": "warning",
      "code": "d",
      "key": "d",
      "image": "docu.png"
    },
    {
      "human": "",
      "command": "not executed",
      "render": "warning",
      "code": "",
      "key": "r",
      "image": ""
    }
  ]
}