Skip to content

Variability4TOSCA Testing 1.0 Release Candidate

This document specifies variability tests for the Variability4TOSCA specification to continuously test that the variability is resolved as expected, e.g., during continuous integration pipelines or before importing the CSAR. The specification is under active development and is not backwards compatible with any previous versions.

Variability Tests

A CSAR might contain variability tests. Therefore, add the directory /tests in the root of the CSAR. Each test is defined inside its own directory of /tests and might contain the following files.

File Description
expected.yaml The expected service template after resolving variability.
inputs.yaml The variability inputs used for resolving variability.
test.yaml The test configuration.

Here is an exemplary structure of a CSAR that has one variability test.

1
2
3
4
5
6
7
<CSAR Name>/
├─ tests/
│  ├─ <Test Name>/
│  │  ├─ expected.yaml
│  │  ├─ inputs.yaml
│  │  ├─ test.yaml
├─ service-template.yaml

The following configurations can be configured in the test configuration file.

Keyname Mandatory Type Description
name false String Display name of the test case (default is the directory name).
description false String Description of the test case.
presets false String | List(String) Variability presets to use when resolving variability.
error false String The expected error that is thrown.
template false String Path (relative to test.yaml) to the variable service template.
expected false String Path (relative to test.yaml) to the expected service template after resolving variability.
inputs false String Path (relative to test.yaml) to the variability inputs.
replace false List(Tuple(Find: String, Replace: String)) An optional list of strings to search and replace for in the generated service template.
merge false Object An optional object that is merged into the expected service template.

There are the following special cases considering test directory names.

  • Hidden Directories: Directories starting with . are ignored.
  • Isolating Tests: Append ---only to a test directory to run only this specific test.
  • Disabling Tests: Append ---disabled to a test directory to disable a specific test.

Example

The following service template contains two conditional nodes. A variability inputs decides which node should be present.

/my-testing-csar/service-template.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
tosca_definitions_version: tosca_variability_1_0

topology_template:
    variability:
        inputs:
            mode:
                type: string

        options:
            type_default_condition: true

    node_templates:
        node_one:
            type: tosca.nodes.Root
            conditions: {equal: [{variability_input: mode}, one]}

        node_two:
            type: tosca.nodes.Root
            conditions: {equal: [{variability_input: mode}, two]}

We specify the following test in which we expect that only the first node is present after resolving variability.

/my-testing-csar/my-test/test.yaml
1
2
3
4
5
6
name: My Test
descriptions: | 
    This tests is a simple example.
    We expect the following: 
    - First node is preserved
    - Second node is removed

The following inputs are used.

/my-testing-csar/my-test/inputs.yaml
1
mode: one

We expect the following service template.

/my-testing-csar/my-test/expected.yaml
1
2
3
4
5
6
tosca_definitions_version: tosca_simple_yaml_1_3

topology_template:
    node_templates:
        node_one:
            type: tosca.nodes.Root

To execute the test using our reference implementation, run the following command.

1
vintner template test --path path/to/my-testing-csar/

Reference Implementation

We provide a reference implementation for this specification as part of OpenTOSCA Vintner. OpenTOSCA Vintner is a TOSCA preprocessing and management layer. The project is open-source, licensed under Apache-2.0, and publicly available at GitHub.

Acknowledgments

This specification is developed for the purpose of research by the Institute of Software Engineering (ISTE) and the Institute of Architecture of Application Systems (IAAS) of the University of Stuttgart, Germany. The development is partially funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) as part of the Software-Defined Car (SofDCar) project (19S21002).

Correspondence

Please address all correspondence concerning this specification to Miles Stötzner <miles.stoetzner@iste.uni-stuttgart.de, https://miles.stoetzner.de>.

Disclaimer of Warranty

Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.

Last update: November 20, 2024