Version 1.4

OpenTestAPI Specification

[Download JSON Schema Specification 1.4]

Disclaimer

Part of this content has been taken from the great work done by the folks at the [AsyncAPI] initiative and the [OpenAPI] initiative.

Introduction

The OpenTestAPI specification is a project to describe and document tests cases in a machine-(and human-)readable format. The specification defines a set of files to describe a test case. Creation and execution of the test cases can be done by utilities.

Schema Specification

OpenTestAPI Object

This is the root document object for the API specification. It combines resource listing and API declaration together into one document.

TestAPI Version String

The version string signifies the version of the TestAPI Specification that the document complies to. The format for this string must be major.minor.

A major.minor shall be used to designate the TestAPI Specification version, and will be considered compatible with the TestAPI Specification specified by that major.minor version.

In subsequent versions of the TestAPI Specification, care will be given such that increments of the minor version should not interfere with operations of tooling developed to a lower minor version. Thus a hypothetical 1.2 specification should be usable with tooling designed for 1.0 (clean handling of allowed values required as minor versions might increase allowed value ranges and add new optional values).

Identifier

This field represents a unique universal identifier. The format is in general free, but it can be used to package test cases to testsuites. Good practice is to follow the URI format, according to RFC3986.

EXAMPLE

{
    "id": "myproject:myexampleapplication:E2E:test1" 
}

{
    "id": "myproject:myexampleapplication:E2E:test1:kafka:inject:mymessage" 
}

Injection Object

Service Object

A SUT can have several interfaces for interaction. The OpenTestAPI defines a common approach to interact with different service interfaces. Therefore, OpenTestingAPI defines a common and a custom configuration part. The custom configuration

The common service configuration part looks as follows

At this moment, the OpenTestAPI supports the following services:

ATTENTION: Do NOT use any technologies effecting business process, for example queues!

Kafka Service Object

Addition / Custom Fields:

EXAMPLE

{
    "service": {
        "type": "kafka",
        "connectstring":"mykafkabroker.com",
        "username": "",
        "password": "",        
        "custom": [
            {
                "key": "topic",
                "value": "mytopic"
            },
            {
                "key": "group.id",
                "value": "mygroup"
            }
        ]             
    }
}

JDBC Service Object

The connectstring for SQL databases follows the JDBC Url syntax. The Url string consists of a protocol-, host-, database- and properties definition.

EXAMPLE

{
  "service": {
        "type": "jdbc",
        "connectstring":"jdbc:oracle:thin:@localhost:1521/XE",
        "username": "myusername",
        "password": "dW5zZWN1cmUgcGFzc3dvcmQ="    
    }
}

Cassandra Service Object

Please use server:port to specify the target cluster.

EXAMPLE

{
  "service": {
        "type": "cassandra",
        "connectstring":"cassandra-db:9042",
        "username": "myusername",
        "password": "dW5zZWN1cmUgcGFzc3dvcmQ="    
    }
}

REST Service Object

Addition / Custom Fields:

EXAMPLE

{
  "service": {
        "type": "rest",
        "connectstring":"http://localhost:50000/dummy/post",
        "username": "myusername",
        "password": "dW5zZWN1cmUgcGFzc3dvcmQ=",
        "custom": [
            {
                "key": "type",
                "value": "post"
            }
        ]   
  }
}

S3 Service Object

Addition / Custom Fields:

EXAMPLE

{
  "service": {
        "type": "s3",
        "connectstring":"s3-serviceEndpoint",
        "username": "myaccesskey",
        "password": "mysecretaccesskey",
        "custom": [
            {
                "key": "type",
                "value": "post"
            }
        ]   
  }
}

Check Object

Validation Object

EXAMPLE

{
  "validations" : [{
        "order": 1,
        "request": "check_postgresql_param_1.sql",
        "type": "containsOneOf",
        "response": [ "expected_result_test1.txt", "expected_result_test2.txt"]        
      }
  ]
}

Checks

A set of checkids, describing the checks executed after the inject was triggered.

EXAMPLE

{
    "checks": [ "kafka:check1", "kafka:check2", "oracle:check1"]
}

Check Type Definition

The Open-Test-API supports the following types of validation:

Hint regex: You might want to use an Online tool first to escape your content before adding real regex, otherwise you will fail with special characters.

Cron Trigger

The Cron Trigger is a comma-separated list of cron jobs. The cron job definition follows the UNIX cronjob definition ([Cron Job])

The following examples will trigger the data inject every hour 5 and 15 minutes after the full hour.

EXAMPLE

{
    "cron" : [ "5 * * * * ?" , "15 * * * * ?"], 
}

Filename

The text content of file will be taken as the input for injects and/or checks. The filename can be name only or relative path to your data source. It’s a good practice to have the files next to your testcase description, e.g. all data sourcefiles for a testcase are inside a directory together with the testcase.
Within test case definition you MUST use the filename only, as implementation is not aware of storage type and folders - upload will use filename only!

EXAMPLE With relative path ./

/testcase1/testcasedescription.json
/testcase1/inject1.txt
/testcase1/inject2.txt
/testcase1/check1.txt
/testcase1/check2.txt

or with relative path testcase1/

testcasedescription.json
/testcase1/inject1.txt
/testcase1/inject2.txt
/testcase1/check1.txt
/testcase1/check2.txt

Time Duration Object

A time duration MUST be a number bigger 0 representing the duration in seconds. For simplification, the following abbreviations are possible:

EXAMPLE

{
  "maxwaitime": "60",
  "maxwaitime": "60s",
  "maxwaitime": "1m",
}

Replacement Object

It’s possible to inject random generated data into the input object and validate the output object for this generated date.

With this, it’s possible to trace objects during a complex SUT overall several subsystems.

A replacement object is a set of

Replacement Rule

Replacement value

The description of a single value looks as follows

The following types are supported by the OpenTestApi Definition

EXAMPLES

A simple timestamp example:

"replacements" : [
{
    "key" : "#key1#",
    "value": [{   
        "order": 1,
        "type": "now",    
        "value": "yyyy-MM-dd HH:mm:ss",
        "param" : "+1h"
    }]   
}]

This will generate a replacement value, which generates a timestamp with one hour in the future. That is, your inject job will be done at 10am, the generated value will be 11am.

"replacements" : [
{
    "key" : "#key1#",
    "value": [{        
        "order": 1,
        "type": "list",    
        "value": "v1|v2|v3|v5"
    }]   
}]

The generator will pick random one of the values out of the list given as param.

"replacements" : [
{
    "key" : "#key1#",
    "value": [{        
        "order": 1,
        "type": "regex",    
        "value": "^[0-9A-F]{8}-[0-9A-F]{4}-4[0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$"
    }],        
}]

This regex generates a UUID in format v4.

A more complex example combines several generators:

"replacements" : [
{
    "key" : "#key1#",
    "value": [{
        "order": 1,
        "type": "list",    
        "value": "v1_0|v2_|v3_|v5_00"
    },{
        "order": 2,
        "type": "regex",    
        "value": "^[0-9]{2}$"
    }],
    "maxlength":5
}]

This will pick a value out of the list “v1_0, v2_, v3_, v5_00” and append a random value between 0 and 99. The result will be cut at length 5, you will end up having only minor version 00 for v5 and minor version 00 to 09 for v1.

YML support (optional)

For readability, we recommend supporting the YML format. However, conversion can also be done as an upstream step, so this feature is optional.