Skip to content

Variability4TOSCA Conformance Test Suite 1.0 Release Candidate

This document specifies the conformance test suite for the Variability4TOSCA specification which can be used to evaluate Variability4TOSCA implementations. The specification is under active development and is not backwards compatible with any previous versions.

Conformance Tests

Each test is described on a separate page. The description includes typically a variable service template, variability inputs, expected errors, expected variability-resolved service template and so on along with an explanation about the expected processing.

ID Test
1 Operator "count"
2 Operator "exponential_regression"
3 Operator "variance"
4 Operator "logarithmic_regression"
5 Operator "max"
6 Operator "mean"
7 Operator "median"
8 Operator "min"
9 Operator "polynomial_regression"
10 Operator "standard_deviation"
11 Operator "sum"
12 Operator "variance"
13 anchor-anchor
14 anchor-anchor-false
15 anchor-anchor-throw
16 anchor-anchor-undefined
17 Operator "add"
18 Operator "div"
19 Operator "mod" (Even)
20 Operator "add" (Odd)
21 Operator "mul"
22 Operator "sub"
23 artifacts-artifact-presence-number-absent
24 artifacts-artifact-presence-number-present
25 artifacts-artifact-presence-string-absent
26 artifacts-artifact-presence-string-present
27 artifacts-conditional
28 artifacts-container-presence-absent
29 artifacts-container-presence-present
30 artifacts-default-alternative
31 artifacts-default-alternative-automatic
32 artifacts-default-alternative-false
33 artifacts-default-condition
34 artifacts-dont-throw-ambiguous-artifact
35 Artifact Pruning
36 artifacts-pruning-managed-manual
37 artifacts-pruning-managed-rules
38 artifacts-pruning-managed-rules-host
39 artifacts-pruning-managed-rules-multiple-different
40 artifacts-pruning-managed-rules-none
41 artifacts-pruning-manged-rules-multiple-same
42 artifacts-required-artifact-constraint-sat
43 artifacts-required-artifact-constraint-sat-pruning
44 artifacts-required-artifact-constraint-unsat
45 artifacts-throw-ambiguous-artifact
46 artifacts-throw-ambiguous-default
47 artifacts-throw-missing-artifact-container
48 artifacts-unique-artifact-constraint-sat-none
49 artifacts-unique-artifact-constraint-sat-one
50 artifacts-unique-artifact-constraint-sat-other
51 artifacts-unique-artifact-constraint-unsat
52 Operator "and" Evaluates to "True" (Empty)
53 Operator "and" Evaluates to "false"
54 Operator "and" Evaluates to "false" (Nested)
55 Operator "and" Evaluates to "true"
56 Operator "and" Evaluates to "true" (Nested)
57 Operator "exo": all false -> false
58 Operator "exo": One ture -> true
59 Operator "exo": all true -> false
60 Operator "implies": false false -> true
61 Operator "implies": false true -> true
62 Operator "implies": true false -> false
63 Operator "implies": true true -> true
64 Operator "not" Evaluates to "false"
65 Operator "not" Evaluates to "true"
66 Operator "or" Evaluates to "False" (Empty)
67 Operator "or" Evaluates to "False"
68 Operator "or" Evaluates to "true"
69 Operator "or" Evaluates to "true" (All "true")
70 circles-alpha
71 circles-bravo
72 circles-charlie
73 circles-delta
74 circles-echo
75 circles-golf
76 circles-hotel
77 circles-india
78 consistency-throw-hosting-relation-missing
79 consistency-throw-hosting-relation-missing-constraint
80 consistency-throw-multiple-hosting-relations
81 consistency-throw-multiple-hosting-relations-constraint
82 consistency-throw-relation-source-missing
83 consistency-throw-relation-target-missing
84 Operator "equal" Evaluates to "false"
85 Operator "equal" Evaluates to "true"
86 Operator "greater_or_equal" Evaluates to "true" (Greater)
87 Operator "greater_or_equal" Evaluates to "true" (Equal)
88 Operator "greater_or_equal" Evaluates to "false" (Less)
89 Operator "greater" Evaluates to "false" (Equal)
90 Operator "greater" Evaluates to "true" (Greater)
91 Operator "greater" Evaluates to "false" (Less)
92 Operator "in_range" Evaluates to "false"
93 Operator "in_range" Evaluates to "true"
94 Operator "length" Evaluates to "false"
95 Operator "length" Evaluates to "true"
96 Operator "less_or_equal" Evaluates to "true" (Equal)
97 Operator "less_or_equal" Evaluates to "false" (Greater)
98 Operator "less_or_equal" Evaluates to "true" (Less)
99 Operator "less" Evaluates to "false" (Equal)
100 Operator "less" Evaluates to "false" (Greater)
101 Operator "less" Evaluates to "true" (Less)
102 Operator "max_length" Evaluates to "true" (Equal)
103 Operator "max_length" Evaluates to "false" (Greater)
104 Operator "max_length" Evaluates to "true" (Less)
105 Operator "min_length" Evaluates to "true" (Equal)
106 Operator "min_length" Evaluates to "true" (Greater)
107 Operator "min_length" Evaluates to "false" (Less)
108 Operator "valid_values" Evaluates to "false"
109 Operator "valid_values" Evaluates to "true"
110 date-operators-after-false
111 date-operators-after-or-same-false
112 date-operators-after-or-same-true
113 date-operators-after-true
114 date-operators-before-false
115 date-operators-before-or-same-false
116 date-operators-before-or-same-true
117 date-operators-before-true
118 date-operators-same-false
119 date-operators-same-true
120 date-operators-within-false
121 date-operators-within-true
122 Conditional Group
123 groups-default-condition
124 groups-default-condition-nothing
125 groups-group-presence-absent
126 groups-group-presence-present
127 groups-prune
128 hosting-aware-pruning-constraints-elastic
129 hosting-aware-pruning-constraints-static
130 hosting-aware-pruning-fig-10-broken
131 hosting-aware-pruning-fig-10-fixed
132 hosting-aware-pruning-host-elastic
133 hosting-aware-pruning-host-static
134 hosting-aware-pruning-motivation-elastic
135 hosting-aware-pruning-motivation-static
136 hosting-aware-pruning-provider-elastic
137 hosting-aware-pruning-provider-static
138 hosting-aware-pruning-transitive-ambiguous
139 hosting-aware-pruning-transitive-both
140 hosting-aware-pruning-transitive-preserved
141 hosting-aware-pruning-transitive-unexpected
142 implications-constraints
143 implications-implied-left
144 implications-implied-right
145 implications-implies-left
146 implications-implies-right
147 implications-incoming-left
148 implications-incoming-right
149 implications-motivating-scenario-cloud
150 implications-motivating-scenario-static
151 implications-requires---disabled
152 implications-unexpected
153 implications-naive-constraints
154 implications-naive-implied-left
155 implications-naive-implied-right
156 implications-naive-implies-left
157 implications-naive-implies-right
158 implications-naive-incoming-left
159 implications-naive-incoming-right
160 implications-naive-motivating-scenario-cloud
161 implications-naive-motivating-scenario-static
162 implications-naive-naive
163 implications-naive-naive-another
164 implications-naive-naive-another-no-optimization
165 implications-naive-naive-another-not
166 implications-naive-naive-not
167 implications-naive-requires---disabled
168 implications-naive-unexpected
169 implied-default-fixed-left
170 implied-default-fixed-right
171 implied-default-manual-left
172 implied-default-manual-right
173 implied-enhanced-new
174 implied-enhanced-old
175 implied-host-broken---disabled
176 implied-host-fixed-left
177 implied-host-fixed-right
178 implied-host-manual-left
179 implied-host-manual-right
180 import-conditional
181 import-empty
182 import-get-import-absent
183 import-get-import-present
184 inputs-conditional
185 inputs-conditional-list
186 inputs-default-alternative
187 inputs-input-presence-absent
188 inputs-input-presence-present
189 inputs-pruning-consumed
190 inputs-pruning-consumed-still
191 inputs-pruning-consumed-v3
192 inputs-pruning-no-consumer
193 inputs-pruning-not-consumed-anymore
194 inputs-pruning-not-consumed-anymore-v3
195 inputs-unconsumed-input-check
196 inputs-unconsumed-input-check-unproduced-output-check
197 inputs-unique-input-constraint-sat
198 inputs-unique-input-constraint-sat-other
199 inputs-unique-input-constraint-sat-pruning-consumed
200 inputs-unique-input-constraint-sat-pruning-consumed-default
201 inputs-unique-input-constraint-unsat
202 nodes-conditional
203 nodes-conditional-type
204 nodes-conditional-type-default
205 nodes-default-false-false
206 nodes-default-false-true
207 nodes-default-host-absent
208 nodes-default-host-present
209 nodes-default-true-false
210 nodes-default-true-true
211 nodes-host-presence-absent
212 nodes-host-presence-false-false
213 nodes-host-presence-present
214 nodes-host-presence-true-false
215 nodes-host-presence-true-true
216 nodes-host-presence-true-true-constraint
217 nodes-node-presence-absent
218 nodes-node-presence-present
219 nodes-required-incoming-relation-constraint-sat
220 nodes-required-incoming-relation-constraint-unsat
221 nodes-type-specific
222 Benchmark
223 Operator "concat"
224 Get Variability Expression
225 Operator "concat"
226 other-logic-expression-false
227 other-logic-expression-true
228 other-logic-value-expression
229 other-nothing
230 other-preset
231 other-presets
232 other-throw-CONTAINER
233 other-throw-SELF
234 other-throw-not-unique
235 Operator "concat"
236 Get Variability Expression
237 Get Variability Input
238 other-version
239 other-weight
240 other-weight-boolean
241 outputs-conditional
242 outputs-conditional-list
243 outputs-default
244 outputs-input-presence-absent
245 outputs-input-presence-present
246 outputs-pruning-not-produced-anymore-unfurl-1
247 outputs-pruning-not-produced-anymore-unfurl-2
248 outputs-pruning-not-produced-anymore-xopera
249 outputs-pruning-not-produced-anymore-xopera-v3
250 outputs-pruning-produced-unfurl
251 outputs-pruning-produced-unfurl-attribute
252 outputs-pruning-produced-unfurl-attribute-not
253 outputs-unique-output-constraint-sat
254 outputs-unique-output-constraint-sat-default
255 outputs-unique-output-constraint-sat-other
256 outputs-unique-output-constraint-sat-pruning-not-produced-anymore-xopera
257 outputs-unique-output-constraint-sat-pruning-produced-xopera
258 outputs-unique-output-constraint-unsat
259 policies-conditional
260 policies-default-condition
261 policies-default-condition-nothing
262 Policy Targets Absent Members
263 Policy Targets Absent Nodes
264 Policy Has No Targets
265 Policy Targets Present Member
266 Policy Targets One Present Node
267 Policy Targets Present Nodes
268 policies-policy-presence-absent
269 policies-policy-presence-present
270 Policy Pruning
271 properties-artifact-container-presence-absent
272 properties-artifact-container-presence-present
273 properties-conditional
274 properties-default-alternative
275 properties-default-condition
276 properties-expression
277 properties-list
278 properties-list-another
279 properties-map
280 properties-map-another
281 properties-node-container-presence-absent
282 properties-node-container-presence-present
283 properties-node-node-presence-CONTAINER-absent
284 properties-node-node-presence-CONTAINER-present
285 properties-node-property-presence-number-absent
286 properties-node-property-presence-number-present
287 properties-properties
288 properties-relation-container-presence-absent
289 properties-relation-container-presence-present
290 properties-throw-ambiguous-property
291 properties-throw-missing-property-container
292 properties-throw-multiple-default
293 properties-unique-property-constraint-sat
294 properties-unique-property-constraint-unsat
295 pruning-alpha-default
296 pruning-alpha-default-naive
297 pruning-alpha-pruning
298 pruning-artifact-default-alpha
299 pruning-artifact-default-bravo
300 pruning-artifact-default-charlie
301 pruning-artifact-default-delta
302 pruning-artifact-default-naive
303 pruning-bravo-default
304 pruning-bravo-pruning
305 pruning-charlie-default
306 pruning-charlie-pruning
307 pruning-default-artifact-1
308 pruning-default-artifact-2
309 pruning-default-artifact-3
310 pruning-default-property-1
311 pruning-default-property-2
312 pruning-default-property-3
313 pruning-delta-default
314 pruning-delta-pruning
315 pruning-echo-default
316 pruning-echo-pruning
317 pruning-foxtrott-default
318 pruning-foxtrott-pruning
319 pruning-hotel-default
320 pruning-hotel-pruning
321 pruning-incoming-artifact-pruning-alpha
322 pruning-incoming-artifact-pruning-bravo
323 pruning-incoming-artifact-pruning-charlie
324 pruning-incoming-artifact-pruning-delta
325 pruning-incoming-artifact-pruning-echo
326 pruning-incoming-artifact-pruning-foxtrott
327 pruning-incoming-artifact-pruning-golf
328 pruning-incoming-artifact-pruning-hotel
329 pruning-incoming-artifact-pruning-naive-alpha
330 pruning-incoming-artifact-pruning-naive-bravo
331 pruning-india-default
332 pruning-india-pruning
333 pruning-juliett-default
334 pruning-juliett-pruning
335 pruning-kilo-default
336 pruning-kilo-pruning
337 pruning-lima-default
338 pruning-lima-pruning
339 pruning-mike-default
340 pruning-mike-pruning
341 pruning-outgoing-relation-absent
342 pruning-outgoing-relation-present
343 pruning-outgoing-target-absent
344 pruning-outgoing-target-present
345 pruning-outgoingnaive-target-present
346 pruning-pruning
347 pruning-pruning-false-false
348 pruning-pruning-false-true
349 pruning-pruning-true-false
350 pruning-pruning-true-true
351 Relation Pruning
352 relationships-ambigous-disabled
353 relationships-ambigous-throw
354 relationships-conditional
355 relationships-throw-multiple
356 relationships-throw-undefined
357 relationships-throw-unused
358 relationships-type-specific
359 requirement-assignment-conditional
360 requirement-assignment-default-alternative
361 requirement-assignment-default-alternative-false
362 requirement-assignment-default-condition
363 requirement-assignment-default-condition-throw-source
364 requirement-assignment-default-condition-throw-target
365 requirement-assignment-one-hosting-relation
366 requirement-assignment-relation-presence-index-absent
367 requirement-assignment-relation-presence-name-absent
368 requirement-assignment-source-presence-absent
369 requirement-assignment-source-presence-present
370 requirement-assignment-target-presence-absent
371 requirement-assignment-target-presence-present
372 requirement-assignment-throw-multiple-defaults
373 Scenario "Alpha"
374 scenario-worker-agent-false
375 scenario-worker-agent-true
376 scenario-worker-worker-false
377 scenario-worker-worker-true
378 technologies-ambiguous
379 technologies-ambiguous-disabled
380 technologies-conditional
381 technologies-constraint
382 technologies-constraint-disabled
383 technologies-default-multiple
384 technologies-default-true
385 technologies-expected
386 technologies-expected-disabled
387 technologies-lowercase
388 technologies-missing
389 technologies-missing-disabled
390 technologies-operations
391 technologies-optimization
392 technologies-optimization-double
393 technologies-optimization-order
394 technologies-optimization-prio
395 technologies-optimization-unsat
396 technologies-optimization-weight
397 technologies-optimization-weight-boolean
398 technologies-pruning
399 technologies-pruning-disabled
400 technologies-rules
401 technologies-rules-v3
402 technologies-throw-required
403 technologies-unique
404 technologies-unique-disabled
405 variability-inputs-excludes
406 variability-inputs-excludes-throw
407 variability-inputs-no-constraints
408 variability-inputs-optional
409 variability-inputs-optional-throw
410 variability-inputs-requires
411 variability-inputs-requires-excludes-targeted
412 variability-inputs-requires-throw
413 variability-inputs-requires-throw-falsy
414 variability-inputs-requires-throw-truthy

Reference Implementation

We provide a reference implementation for this specification as part of OpenTOSCA Vintner. OpenTOSCA Vintner is a TOSCA preprocessing and management layer. The project is open-source, licensed under Apache-2.0, and publicly available at GitHub.

Acknowledgments

This specification is developed for the purpose of research by the Institute of Software Engineering (ISTE) and the Institute of Architecture of Application Systems (IAAS) of the University of Stuttgart, Germany. The development is partially funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) as part of the Software-Defined Car (SofDCar) project (19S21002).

Correspondence

Please address all correspondence concerning this specification to Miles Stötzner <miles.stoetzner@iste.uni-stuttgart.de, https://miles.stoetzner.de>.

Disclaimer of Warranty

Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.