Skip to content

Variability4TOSCA Conformance Test Suite 1.0 Release Candidate

This document specifies the conformance test suite for the Variability4TOSCA specification which can be used to evaluate Variability4TOSCA implementations. The specification is under active development and is not backwards compatible with any previous versions.

Conformance Tests

Each test is described on a separate page. The description includes typically a variable service template, variability inputs, expected errors, expected variability-resolved service template and so on along with an explanation about the expected processing.

ID Test
1 Operator "count"
2 Operator "exponential_regression"
3 Operator "variance"
4 Operator "logarithmic_regression"
5 Operator "max"
6 Operator "mean"
7 Operator "median"
8 Operator "min"
9 Operator "polynomial_regression"
10 Operator "standard_deviation"
11 Operator "sum"
12 Operator "variance"
13 Operator "add"
14 Operator "div"
15 Operator "mod" (Even)
16 Operator "add" (Odd)
17 Operator "mul"
18 Operator "sub"
19 artifacts-artifact-presence-number-absent
20 artifacts-artifact-presence-number-present
21 artifacts-artifact-presence-string-absent
22 artifacts-artifact-presence-string-present
23 artifacts-conditional
24 artifacts-container-presence-absent
25 artifacts-container-presence-present
26 artifacts-default-alternative
27 artifacts-default-alternative-false
28 artifacts-default-condition
29 artifacts-dont-throw-ambiguous-artifact
30 Artifact Pruning
31 artifacts-pruning-managed-manual
32 artifacts-pruning-managed-rules
33 artifacts-pruning-managed-rules-host
34 artifacts-pruning-managed-rules-multiple-different
35 artifacts-pruning-managed-rules-none
36 artifacts-pruning-manged-rules-multiple-same
37 artifacts-required-artifact-constraint-sat
38 artifacts-required-artifact-constraint-sat-pruning
39 artifacts-required-artifact-constraint-unsat
40 artifacts-throw-ambiguous-artifact
41 artifacts-throw-ambiguous-default
42 artifacts-throw-missing-artifact-container
43 artifacts-unique-artifact-constraint-sat-none
44 artifacts-unique-artifact-constraint-sat-one
45 artifacts-unique-artifact-constraint-sat-other
46 artifacts-unique-artifact-constraint-unsat
47 Operator "and" Evaluates to "True" (Empty)
48 Operator "and" Evaluates to "false"
49 Operator "and" Evaluates to "false" (Nested)
50 Operator "and" Evaluates to "true"
51 Operator "and" Evaluates to "true" (Nested)
52 Operator "exo": all false -> false
53 Operator "exo": One ture -> true
54 Operator "exo": all true -> false
55 Operator "implies": false false -> true
56 Operator "implies": false true -> true
57 Operator "implies": true false -> false
58 Operator "implies": true true -> true
59 Operator "not" Evaluates to "false"
60 Operator "not" Evaluates to "true"
61 Operator "or" Evaluates to "False" (Empty)
62 Operator "or" Evaluates to "False"
63 Operator "or" Evaluates to "true"
64 Operator "or" Evaluates to "true" (All "true")
65 circles-alpha
66 circles-bravo
67 circles-charlie
68 circles-delta
69 circles-echo
70 circles-golf
71 circles-hotel
72 circles-india
73 consistency-throw-hosting-relation-missing
74 consistency-throw-hosting-relation-missing-constraint
75 consistency-throw-multiple-hosting-relations
76 consistency-throw-multiple-hosting-relations-constraint
77 consistency-throw-relation-source-missing
78 consistency-throw-relation-target-missing
79 Operator "equal" Evaluates to "false"
80 Operator "equal" Evaluates to "true"
81 Operator "greater_or_equal" Evaluates to "true" (Greater)
82 Operator "greater_or_equal" Evaluates to "true" (Equal)
83 Operator "greater_or_equal" Evaluates to "false" (Less)
84 Operator "greater" Evaluates to "false" (Equal)
85 Operator "greater" Evaluates to "true" (Greater)
86 Operator "greater" Evaluates to "false" (Less)
87 Operator "in_range" Evaluates to "false"
88 Operator "in_range" Evaluates to "true"
89 Operator "length" Evaluates to "false"
90 Operator "length" Evaluates to "true"
91 Operator "less_or_equal" Evaluates to "true" (Equal)
92 Operator "less_or_equal" Evaluates to "false" (Greater)
93 Operator "less_or_equal" Evaluates to "true" (Less)
94 Operator "less" Evaluates to "false" (Equal)
95 Operator "less" Evaluates to "false" (Greater)
96 Operator "less" Evaluates to "true" (Less)
97 Operator "max_length" Evaluates to "true" (Equal)
98 Operator "max_length" Evaluates to "false" (Greater)
99 Operator "max_length" Evaluates to "true" (Less)
100 Operator "min_length" Evaluates to "true" (Equal)
101 Operator "min_length" Evaluates to "true" (Greater)
102 Operator "min_length" Evaluates to "false" (Less)
103 Operator "valid_values" Evaluates to "false"
104 Operator "valid_values" Evaluates to "true"
105 date-operators-after-false
106 date-operators-after-or-same-false
107 date-operators-after-or-same-true
108 date-operators-after-true
109 date-operators-before-false
110 date-operators-before-or-same-false
111 date-operators-before-or-same-true
112 date-operators-before-true
113 date-operators-same-false
114 date-operators-same-true
115 date-operators-within-false
116 date-operators-within-true
117 Conditional Group
118 groups-default-condition
119 groups-default-condition-nothing
120 groups-group-presence-absent
121 groups-group-presence-present
122 groups-prune
123 hosting-aware-pruning-constraints-elastic
124 hosting-aware-pruning-constraints-static
125 hosting-aware-pruning-fig-10-broken
126 hosting-aware-pruning-fig-10-fixed
127 hosting-aware-pruning-host-elastic
128 hosting-aware-pruning-host-static
129 hosting-aware-pruning-motivation-elastic
130 hosting-aware-pruning-motivation-static
131 hosting-aware-pruning-provider-elastic
132 hosting-aware-pruning-provider-static
133 hosting-aware-pruning-transitive-ambiguous
134 hosting-aware-pruning-transitive-both
135 hosting-aware-pruning-transitive-preserved
136 hosting-aware-pruning-transitive-unexpected
137 implications-constraints
138 implications-implied-left
139 implications-implied-right
140 implications-implies-left
141 implications-implies-right
142 implications-incoming-left
143 implications-incoming-right
144 implications-motivating-scenario-cloud
145 implications-motivating-scenario-static
146 implications-requires---disabled
147 implications-unexpected
148 implications-naive-constraints
149 implications-naive-implied-left
150 implications-naive-implied-right
151 implications-naive-implies-left
152 implications-naive-implies-right
153 implications-naive-incoming-left
154 implications-naive-incoming-right
155 implications-naive-motivating-scenario-cloud
156 implications-naive-motivating-scenario-static
157 implications-naive-naive
158 implications-naive-naive-another
159 implications-naive-naive-another-no-optimization
160 implications-naive-naive-another-not
161 implications-naive-naive-not
162 implications-naive-requires---disabled
163 implications-naive-unexpected
164 implied-default-fixed-left
165 implied-default-fixed-right
166 implied-default-manual-left
167 implied-default-manual-right
168 implied-enhanced-new
169 implied-enhanced-old
170 implied-host-broken---disabled
171 implied-host-fixed-left
172 implied-host-fixed-right
173 implied-host-manual-left
174 implied-host-manual-right
175 import-conditional
176 import-empty
177 import-get-import-absent
178 import-get-import-present
179 inputs-conditional
180 inputs-conditional-list
181 inputs-default-alternative
182 inputs-input-presence-absent
183 inputs-input-presence-present
184 inputs-pruning-consumed
185 inputs-pruning-consumed-still
186 inputs-pruning-consumed-v3
187 inputs-pruning-no-consumer
188 inputs-pruning-not-consumed-anymore
189 inputs-pruning-not-consumed-anymore-v3
190 inputs-unique-input-constraint-sat
191 inputs-unique-input-constraint-sat-other
192 inputs-unique-input-constraint-sat-pruning-consumed
193 inputs-unique-input-constraint-sat-pruning-consumed-default
194 inputs-unique-input-constraint-unsat
195 nodes-conditional
196 nodes-conditional-type
197 nodes-conditional-type-default
198 nodes-default-false-false
199 nodes-default-false-true
200 nodes-default-host-absent
201 nodes-default-host-present
202 nodes-default-true-false
203 nodes-default-true-true
204 nodes-host-presence-absent
205 nodes-host-presence-false-false
206 nodes-host-presence-present
207 nodes-host-presence-true-false
208 nodes-host-presence-true-true
209 nodes-host-presence-true-true-constraint
210 nodes-node-presence-absent
211 nodes-node-presence-present
212 nodes-required-incoming-relation-constraint-sat
213 nodes-required-incoming-relation-constraint-unsat
214 nodes-type-specific
215 Benchmark
216 Operator "concat"
217 Get Variability Expression
218 Operator "concat"
219 other-logic-expression-false
220 other-logic-expression-true
221 other-logic-value-expression
222 other-nothing
223 other-preset
224 other-presets
225 other-throw-CONTAINER
226 other-throw-SELF
227 other-throw-not-unique
228 Operator "concat"
229 Get Variability Expression
230 Get Variability Input
231 other-version
232 other-weight
233 outputs-conditional
234 outputs-conditional-list
235 outputs-default
236 outputs-input-presence-absent
237 outputs-input-presence-present
238 outputs-pruning-not-produced-anymore-unfurl-1
239 outputs-pruning-not-produced-anymore-unfurl-2
240 outputs-pruning-not-produced-anymore-xopera
241 outputs-pruning-not-produced-anymore-xopera-v3
242 outputs-pruning-produced-unfurl
243 outputs-pruning-produced-unfurl-attribute
244 outputs-pruning-produced-unfurl-attribute-not
245 outputs-unique-output-constraint-sat
246 outputs-unique-output-constraint-sat-default
247 outputs-unique-output-constraint-sat-other
248 outputs-unique-output-constraint-sat-pruning-not-produced-anymore-xopera
249 outputs-unique-output-constraint-sat-pruning-produced-xopera
250 outputs-unique-output-constraint-unsat
251 persistent-persistent
252 persistent-persistent-false
253 persistent-persistent-throw
254 persistent-persistent-undefined
255 policies-conditional
256 policies-default-condition
257 policies-default-condition-nothing
258 Policy Targets Absent Members
259 Policy Targets Absent Nodes
260 Policy Has No Targets
261 Policy Targets Present Member
262 Policy Targets One Present Node
263 Policy Targets Present Nodes
264 policies-policy-presence-absent
265 policies-policy-presence-present
266 Policy Pruning
267 properties-artifact-container-presence-absent
268 properties-artifact-container-presence-present
269 properties-conditional
270 properties-default-alternative
271 properties-default-alternative-false
272 properties-default-condition
273 properties-expression
274 properties-list
275 properties-list-another
276 properties-map
277 properties-map-another
278 properties-node-container-presence-absent
279 properties-node-container-presence-present
280 properties-node-node-presence-CONTAINER-absent
281 properties-node-node-presence-CONTAINER-present
282 properties-node-property-presence-number-absent
283 properties-node-property-presence-number-present
284 properties-properties
285 properties-relation-container-presence-absent
286 properties-relation-container-presence-present
287 properties-throw-ambiguous-property
288 properties-throw-missing-property-container
289 properties-throw-multiple-default
290 properties-unique-property-constraint-sat
291 properties-unique-property-constraint-unsat
292 pruning-alpha-default
293 pruning-alpha-default-naive
294 pruning-alpha-pruning
295 pruning-artifact-default-alpha
296 pruning-artifact-default-bravo
297 pruning-artifact-default-charlie
298 pruning-artifact-default-delta
299 pruning-artifact-default-naive
300 pruning-bravo-default
301 pruning-bravo-pruning
302 pruning-charlie-default
303 pruning-charlie-pruning
304 pruning-delta-default
305 pruning-delta-pruning
306 pruning-echo-default
307 pruning-echo-pruning
308 pruning-foxtrott-default
309 pruning-foxtrott-pruning
310 pruning-hotel-default
311 pruning-hotel-pruning
312 pruning-incoming-artifact-pruning-alpha
313 pruning-incoming-artifact-pruning-bravo
314 pruning-incoming-artifact-pruning-charlie
315 pruning-incoming-artifact-pruning-delta
316 pruning-incoming-artifact-pruning-echo
317 pruning-incoming-artifact-pruning-foxtrott
318 pruning-incoming-artifact-pruning-golf
319 pruning-incoming-artifact-pruning-hotel
320 pruning-incoming-artifact-pruning-naive-alpha
321 pruning-incoming-artifact-pruning-naive-bravo
322 pruning-india-default
323 pruning-india-pruning
324 pruning-juliett-default
325 pruning-juliett-pruning
326 pruning-kilo-default
327 pruning-kilo-pruning
328 pruning-lima-default
329 pruning-lima-pruning
330 pruning-mike-default
331 pruning-mike-pruning
332 pruning-outgoing-relation-absent
333 pruning-outgoing-relation-present
334 pruning-outgoing-target-absent
335 pruning-outgoing-target-present
336 pruning-outgoingnaive-target-present
337 pruning-pruning
338 pruning-pruning-false-false
339 pruning-pruning-false-true
340 pruning-pruning-true-false
341 pruning-pruning-true-true
342 Relation Pruning
343 relationships-ambigous-disabled
344 relationships-ambigous-throw
345 relationships-conditional
346 relationships-throw-multiple
347 relationships-throw-undefined
348 relationships-throw-unused
349 relationships-type-specific
350 requirement-assignment-conditional
351 requirement-assignment-default-alternative
352 requirement-assignment-default-alternative-false
353 requirement-assignment-default-condition
354 requirement-assignment-default-condition-throw-source
355 requirement-assignment-default-condition-throw-target
356 requirement-assignment-one-hosting-relation
357 requirement-assignment-relation-presence-index-absent
358 requirement-assignment-relation-presence-name-absent
359 requirement-assignment-source-presence-absent
360 requirement-assignment-source-presence-present
361 requirement-assignment-target-presence-absent
362 requirement-assignment-target-presence-present
363 requirement-assignment-throw-multiple-defaults
364 Scenario "Alpha"
365 scenario-worker-agent-false
366 scenario-worker-agent-true
367 scenario-worker-worker-false
368 scenario-worker-worker-true
369 technologies-ambiguous
370 technologies-ambiguous-disabled
371 technologies-conditional
372 technologies-constraint
373 technologies-constraint-disabled
374 technologies-default-multiple
375 technologies-default-true
376 technologies-expected
377 technologies-expected-disabled
378 technologies-lowercase
379 technologies-missing
380 technologies-missing-disabled
381 technologies-optimization
382 technologies-optimization-double
383 technologies-optimization-order
384 technologies-optimization-prio
385 technologies-optimization-unsat
386 technologies-optimization-weight
387 technologies-pruning
388 technologies-pruning-disabled
389 technologies-rules
390 technologies-rules-v3
391 technologies-throw-required
392 technologies-unique
393 technologies-unique-disabled
394 variability-inputs-no-constraints
395 variability-inputs-requires
396 variability-inputs-requires-throw
397 variability-inputs-requires-throw-falsy
398 variability-inputs-requires-throw-truthy

Reference Implementation

We provide a reference implementation for this specification as part of OpenTOSCA Vintner. OpenTOSCA Vintner is a TOSCA preprocessing and management layer. The project is open-source, licensed under Apache-2.0, and publicly available at GitHub.

Acknowledgments

This specification is developed for the purpose of research by the Institute of Software Engineering (ISTE) and the Institute of Architecture of Application Systems (IAAS) of the University of Stuttgart, Germany. The development is partially funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) as part of the Software-Defined Car (SofDCar) project (19S21002).

Correspondence

Please address all correspondence concerning this specification to Miles Stötzner <miles.stoetzner@iste.uni-stuttgart.de, https://miles.stoetzner.de>.

Disclaimer of Warranty

Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.


Last update: January 8, 2025