Skip to content

Variability4TOSCA Conformance Test Suite 1.0 Release Candidate

This document specifies the conformance test suite for the Variability4TOSCA specification which can be used to evaluate Variability4TOSCA implementations. The specification is under active development and is not backwards compatible with any previous versions.

Conformance Tests

Each test is described on a separate page. The description includes typically a variable service template, variability inputs, expected errors, expected variability-resolved service template and so on along with an explanation about the expected processing.

ID Test
1 Operator "count"
2 Operator "exponential_regression"
3 Operator "variance"
4 Operator "logarithmic_regression"
5 Operator "max"
6 Operator "mean"
7 Operator "median"
8 Operator "min"
9 Operator "polynomial_regression"
10 Operator "standard_deviation"
11 Operator "sum"
12 Operator "variance"
13 anchor-anchor
14 anchor-anchor-false
15 anchor-anchor-throw
16 anchor-anchor-undefined
17 Operator "add"
18 Operator "div"
19 Operator "mod" (Even)
20 Operator "add" (Odd)
21 Operator "mul"
22 Operator "sub"
23 artifacts-artifact-presence-number-absent
24 artifacts-artifact-presence-number-present
25 artifacts-artifact-presence-string-absent
26 artifacts-artifact-presence-string-present
27 artifacts-conditional
28 artifacts-container-presence-absent
29 artifacts-container-presence-present
30 artifacts-default-alternative
31 artifacts-default-alternative-false
32 artifacts-default-condition
33 artifacts-dont-throw-ambiguous-artifact
34 Artifact Pruning
35 artifacts-pruning-managed-manual
36 artifacts-pruning-managed-rules
37 artifacts-pruning-managed-rules-host
38 artifacts-pruning-managed-rules-multiple-different
39 artifacts-pruning-managed-rules-none
40 artifacts-pruning-manged-rules-multiple-same
41 artifacts-required-artifact-constraint-sat
42 artifacts-required-artifact-constraint-sat-pruning
43 artifacts-required-artifact-constraint-unsat
44 artifacts-throw-ambiguous-artifact
45 artifacts-throw-ambiguous-default
46 artifacts-throw-missing-artifact-container
47 artifacts-unique-artifact-constraint-sat-none
48 artifacts-unique-artifact-constraint-sat-one
49 artifacts-unique-artifact-constraint-sat-other
50 artifacts-unique-artifact-constraint-unsat
51 Operator "and" Evaluates to "True" (Empty)
52 Operator "and" Evaluates to "false"
53 Operator "and" Evaluates to "false" (Nested)
54 Operator "and" Evaluates to "true"
55 Operator "and" Evaluates to "true" (Nested)
56 Operator "exo": all false -> false
57 Operator "exo": One ture -> true
58 Operator "exo": all true -> false
59 Operator "implies": false false -> true
60 Operator "implies": false true -> true
61 Operator "implies": true false -> false
62 Operator "implies": true true -> true
63 Operator "not" Evaluates to "false"
64 Operator "not" Evaluates to "true"
65 Operator "or" Evaluates to "False" (Empty)
66 Operator "or" Evaluates to "False"
67 Operator "or" Evaluates to "true"
68 Operator "or" Evaluates to "true" (All "true")
69 circles-alpha
70 circles-bravo
71 circles-charlie
72 circles-delta
73 circles-echo
74 circles-golf
75 circles-hotel
76 circles-india
77 consistency-throw-hosting-relation-missing
78 consistency-throw-hosting-relation-missing-constraint
79 consistency-throw-multiple-hosting-relations
80 consistency-throw-multiple-hosting-relations-constraint
81 consistency-throw-relation-source-missing
82 consistency-throw-relation-target-missing
83 Operator "equal" Evaluates to "false"
84 Operator "equal" Evaluates to "true"
85 Operator "greater_or_equal" Evaluates to "true" (Greater)
86 Operator "greater_or_equal" Evaluates to "true" (Equal)
87 Operator "greater_or_equal" Evaluates to "false" (Less)
88 Operator "greater" Evaluates to "false" (Equal)
89 Operator "greater" Evaluates to "true" (Greater)
90 Operator "greater" Evaluates to "false" (Less)
91 Operator "in_range" Evaluates to "false"
92 Operator "in_range" Evaluates to "true"
93 Operator "length" Evaluates to "false"
94 Operator "length" Evaluates to "true"
95 Operator "less_or_equal" Evaluates to "true" (Equal)
96 Operator "less_or_equal" Evaluates to "false" (Greater)
97 Operator "less_or_equal" Evaluates to "true" (Less)
98 Operator "less" Evaluates to "false" (Equal)
99 Operator "less" Evaluates to "false" (Greater)
100 Operator "less" Evaluates to "true" (Less)
101 Operator "max_length" Evaluates to "true" (Equal)
102 Operator "max_length" Evaluates to "false" (Greater)
103 Operator "max_length" Evaluates to "true" (Less)
104 Operator "min_length" Evaluates to "true" (Equal)
105 Operator "min_length" Evaluates to "true" (Greater)
106 Operator "min_length" Evaluates to "false" (Less)
107 Operator "valid_values" Evaluates to "false"
108 Operator "valid_values" Evaluates to "true"
109 date-operators-after-false
110 date-operators-after-or-same-false
111 date-operators-after-or-same-true
112 date-operators-after-true
113 date-operators-before-false
114 date-operators-before-or-same-false
115 date-operators-before-or-same-true
116 date-operators-before-true
117 date-operators-same-false
118 date-operators-same-true
119 date-operators-within-false
120 date-operators-within-true
121 Conditional Group
122 groups-default-condition
123 groups-default-condition-nothing
124 groups-group-presence-absent
125 groups-group-presence-present
126 groups-prune
127 hosting-aware-pruning-constraints-elastic
128 hosting-aware-pruning-constraints-static
129 hosting-aware-pruning-fig-10-broken
130 hosting-aware-pruning-fig-10-fixed
131 hosting-aware-pruning-host-elastic
132 hosting-aware-pruning-host-static
133 hosting-aware-pruning-motivation-elastic
134 hosting-aware-pruning-motivation-static
135 hosting-aware-pruning-provider-elastic
136 hosting-aware-pruning-provider-static
137 hosting-aware-pruning-transitive-ambiguous
138 hosting-aware-pruning-transitive-both
139 hosting-aware-pruning-transitive-preserved
140 hosting-aware-pruning-transitive-unexpected
141 implications-constraints
142 implications-implied-left
143 implications-implied-right
144 implications-implies-left
145 implications-implies-right
146 implications-incoming-left
147 implications-incoming-right
148 implications-motivating-scenario-cloud
149 implications-motivating-scenario-static
150 implications-requires---disabled
151 implications-unexpected
152 implications-naive-constraints
153 implications-naive-implied-left
154 implications-naive-implied-right
155 implications-naive-implies-left
156 implications-naive-implies-right
157 implications-naive-incoming-left
158 implications-naive-incoming-right
159 implications-naive-motivating-scenario-cloud
160 implications-naive-motivating-scenario-static
161 implications-naive-naive
162 implications-naive-naive-another
163 implications-naive-naive-another-no-optimization
164 implications-naive-naive-another-not
165 implications-naive-naive-not
166 implications-naive-requires---disabled
167 implications-naive-unexpected
168 implied-default-fixed-left
169 implied-default-fixed-right
170 implied-default-manual-left
171 implied-default-manual-right
172 implied-enhanced-new
173 implied-enhanced-old
174 implied-host-broken---disabled
175 implied-host-fixed-left
176 implied-host-fixed-right
177 implied-host-manual-left
178 implied-host-manual-right
179 import-conditional
180 import-empty
181 import-get-import-absent
182 import-get-import-present
183 inputs-conditional
184 inputs-conditional-list
185 inputs-default-alternative
186 inputs-input-presence-absent
187 inputs-input-presence-present
188 inputs-pruning-consumed
189 inputs-pruning-consumed-still
190 inputs-pruning-consumed-v3
191 inputs-pruning-no-consumer
192 inputs-pruning-not-consumed-anymore
193 inputs-pruning-not-consumed-anymore-v3
194 inputs-unique-input-constraint-sat
195 inputs-unique-input-constraint-sat-other
196 inputs-unique-input-constraint-sat-pruning-consumed
197 inputs-unique-input-constraint-sat-pruning-consumed-default
198 inputs-unique-input-constraint-unsat
199 nodes-conditional
200 nodes-conditional-type
201 nodes-conditional-type-default
202 nodes-default-false-false
203 nodes-default-false-true
204 nodes-default-host-absent
205 nodes-default-host-present
206 nodes-default-true-false
207 nodes-default-true-true
208 nodes-host-presence-absent
209 nodes-host-presence-false-false
210 nodes-host-presence-present
211 nodes-host-presence-true-false
212 nodes-host-presence-true-true
213 nodes-host-presence-true-true-constraint
214 nodes-node-presence-absent
215 nodes-node-presence-present
216 nodes-required-incoming-relation-constraint-sat
217 nodes-required-incoming-relation-constraint-unsat
218 nodes-type-specific
219 Benchmark
220 Operator "concat"
221 Get Variability Expression
222 Operator "concat"
223 other-logic-expression-false
224 other-logic-expression-true
225 other-logic-value-expression
226 other-nothing
227 other-preset
228 other-presets
229 other-throw-CONTAINER
230 other-throw-SELF
231 other-throw-not-unique
232 Operator "concat"
233 Get Variability Expression
234 Get Variability Input
235 other-version
236 other-weight
237 outputs-conditional
238 outputs-conditional-list
239 outputs-default
240 outputs-input-presence-absent
241 outputs-input-presence-present
242 outputs-pruning-not-produced-anymore-unfurl-1
243 outputs-pruning-not-produced-anymore-unfurl-2
244 outputs-pruning-not-produced-anymore-xopera
245 outputs-pruning-not-produced-anymore-xopera-v3
246 outputs-pruning-produced-unfurl
247 outputs-pruning-produced-unfurl-attribute
248 outputs-pruning-produced-unfurl-attribute-not
249 outputs-unique-output-constraint-sat
250 outputs-unique-output-constraint-sat-default
251 outputs-unique-output-constraint-sat-other
252 outputs-unique-output-constraint-sat-pruning-not-produced-anymore-xopera
253 outputs-unique-output-constraint-sat-pruning-produced-xopera
254 outputs-unique-output-constraint-unsat
255 policies-conditional
256 policies-default-condition
257 policies-default-condition-nothing
258 Policy Targets Absent Members
259 Policy Targets Absent Nodes
260 Policy Has No Targets
261 Policy Targets Present Member
262 Policy Targets One Present Node
263 Policy Targets Present Nodes
264 policies-policy-presence-absent
265 policies-policy-presence-present
266 Policy Pruning
267 properties-artifact-container-presence-absent
268 properties-artifact-container-presence-present
269 properties-conditional
270 properties-default-alternative
271 properties-default-alternative-false
272 properties-default-condition
273 properties-expression
274 properties-list
275 properties-list-another
276 properties-map
277 properties-map-another
278 properties-node-container-presence-absent
279 properties-node-container-presence-present
280 properties-node-node-presence-CONTAINER-absent
281 properties-node-node-presence-CONTAINER-present
282 properties-node-property-presence-number-absent
283 properties-node-property-presence-number-present
284 properties-properties
285 properties-relation-container-presence-absent
286 properties-relation-container-presence-present
287 properties-throw-ambiguous-property
288 properties-throw-missing-property-container
289 properties-throw-multiple-default
290 properties-unique-property-constraint-sat
291 properties-unique-property-constraint-unsat
292 pruning-alpha-default
293 pruning-alpha-default-naive
294 pruning-alpha-pruning
295 pruning-artifact-default-alpha
296 pruning-artifact-default-bravo
297 pruning-artifact-default-charlie
298 pruning-artifact-default-delta
299 pruning-artifact-default-naive
300 pruning-bravo-default
301 pruning-bravo-pruning
302 pruning-charlie-default
303 pruning-charlie-pruning
304 pruning-delta-default
305 pruning-delta-pruning
306 pruning-echo-default
307 pruning-echo-pruning
308 pruning-foxtrott-default
309 pruning-foxtrott-pruning
310 pruning-hotel-default
311 pruning-hotel-pruning
312 pruning-incoming-artifact-pruning-alpha
313 pruning-incoming-artifact-pruning-bravo
314 pruning-incoming-artifact-pruning-charlie
315 pruning-incoming-artifact-pruning-delta
316 pruning-incoming-artifact-pruning-echo
317 pruning-incoming-artifact-pruning-foxtrott
318 pruning-incoming-artifact-pruning-golf
319 pruning-incoming-artifact-pruning-hotel
320 pruning-incoming-artifact-pruning-naive-alpha
321 pruning-incoming-artifact-pruning-naive-bravo
322 pruning-india-default
323 pruning-india-pruning
324 pruning-juliett-default
325 pruning-juliett-pruning
326 pruning-kilo-default
327 pruning-kilo-pruning
328 pruning-lima-default
329 pruning-lima-pruning
330 pruning-mike-default
331 pruning-mike-pruning
332 pruning-outgoing-relation-absent
333 pruning-outgoing-relation-present
334 pruning-outgoing-target-absent
335 pruning-outgoing-target-present
336 pruning-outgoingnaive-target-present
337 pruning-pruning
338 pruning-pruning-false-false
339 pruning-pruning-false-true
340 pruning-pruning-true-false
341 pruning-pruning-true-true
342 Relation Pruning
343 relationships-ambigous-disabled
344 relationships-ambigous-throw
345 relationships-conditional
346 relationships-throw-multiple
347 relationships-throw-undefined
348 relationships-throw-unused
349 relationships-type-specific
350 requirement-assignment-conditional
351 requirement-assignment-default-alternative
352 requirement-assignment-default-alternative-false
353 requirement-assignment-default-condition
354 requirement-assignment-default-condition-throw-source
355 requirement-assignment-default-condition-throw-target
356 requirement-assignment-one-hosting-relation
357 requirement-assignment-relation-presence-index-absent
358 requirement-assignment-relation-presence-name-absent
359 requirement-assignment-source-presence-absent
360 requirement-assignment-source-presence-present
361 requirement-assignment-target-presence-absent
362 requirement-assignment-target-presence-present
363 requirement-assignment-throw-multiple-defaults
364 Scenario "Alpha"
365 scenario-worker-agent-false
366 scenario-worker-agent-true
367 scenario-worker-worker-false
368 scenario-worker-worker-true
369 technologies-ambiguous
370 technologies-ambiguous-disabled
371 technologies-conditional
372 technologies-constraint
373 technologies-constraint-disabled
374 technologies-default-multiple
375 technologies-default-true
376 technologies-expected
377 technologies-expected-disabled
378 technologies-lowercase
379 technologies-missing
380 technologies-missing-disabled
381 technologies-optimization
382 technologies-optimization-double
383 technologies-optimization-order
384 technologies-optimization-prio
385 technologies-optimization-unsat
386 technologies-optimization-weight
387 technologies-pruning
388 technologies-pruning-disabled
389 technologies-rules
390 technologies-rules-v3
391 technologies-throw-required
392 technologies-unique
393 technologies-unique-disabled
394 variability-inputs-no-constraints
395 variability-inputs-requires
396 variability-inputs-requires-throw
397 variability-inputs-requires-throw-falsy
398 variability-inputs-requires-throw-truthy

Reference Implementation

We provide a reference implementation for this specification as part of OpenTOSCA Vintner. OpenTOSCA Vintner is a TOSCA preprocessing and management layer. The project is open-source, licensed under Apache-2.0, and publicly available at GitHub.

Acknowledgments

This specification is developed for the purpose of research by the Institute of Software Engineering (ISTE) and the Institute of Architecture of Application Systems (IAAS) of the University of Stuttgart, Germany. The development is partially funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) as part of the Software-Defined Car (SofDCar) project (19S21002).

Correspondence

Please address all correspondence concerning this specification to Miles Stötzner <miles.stoetzner@iste.uni-stuttgart.de, https://miles.stoetzner.de>.

Disclaimer of Warranty

Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.


Last update: April 23, 2025