Skip to content

Variability4TOSCA Conformance Test Suite 1.0 Release Candidate

This document specifies the conformance test suite for the Variability4TOSCA specification which can be used to evaluate Variability4TOSCA implementations. The specification is under active development and is not backwards compatible with any previous versions.

Conformance Tests

Each test is described on a separate page. The description includes typically a variable service template, variability inputs, expected errors, expected variability-resolved service template and so on along with an explanation about the expected processing.

ID Test
1 Operator "count"
2 Operator "exponential_regression"
3 Operator "variance"
4 Operator "logarithmic_regression"
5 Operator "max"
6 Operator "mean"
7 Operator "median"
8 Operator "min"
9 Operator "polynomial_regression"
10 Operator "standard_deviation"
11 Operator "sum"
12 Operator "variance"
13 Operator "add"
14 Operator "div"
15 Operator "mod" (Even)
16 Operator "add" (Odd)
17 Operator "mul"
18 Operator "sub"
19 artifacts-artifact-presence-number-absent
20 artifacts-artifact-presence-number-present
21 artifacts-artifact-presence-string-absent
22 artifacts-artifact-presence-string-present
23 artifacts-conditional
24 artifacts-container-presence-absent
25 artifacts-container-presence-present
26 artifacts-default-alternative
27 artifacts-default-alternative-false
28 artifacts-default-condition
29 artifacts-dont-throw-ambiguous-artifact
30 Artifact Pruning
31 artifacts-pruning-managed-manual
32 artifacts-pruning-managed-rules
33 artifacts-pruning-managed-rules-host
34 artifacts-pruning-managed-rules-multiple-different
35 artifacts-pruning-managed-rules-none
36 artifacts-pruning-manged-rules-multiple-same
37 artifacts-required-artifact-constraint-sat
38 artifacts-required-artifact-constraint-sat-pruning
39 artifacts-required-artifact-constraint-unsat
40 artifacts-throw-ambiguous-artifact
41 artifacts-throw-ambiguous-default
42 artifacts-throw-missing-artifact-container
43 artifacts-unique-artifact-constraint-sat-none
44 artifacts-unique-artifact-constraint-sat-one
45 artifacts-unique-artifact-constraint-sat-other
46 artifacts-unique-artifact-constraint-unsat
47 Operator "and" Evaluates to "True" (Empty)
48 Operator "and" Evaluates to "false"
49 Operator "and" Evaluates to "false" (Nested)
50 Operator "and" Evaluates to "true"
51 Operator "and" Evaluates to "true" (Nested)
52 Operator "exo": all false -> false
53 Operator "exo": One ture -> true
54 Operator "exo": all true -> false
55 Operator "implies": false false -> true
56 Operator "implies": false true -> true
57 Operator "implies": true false -> false
58 Operator "implies": true true -> true
59 Operator "not" Evaluates to "false"
60 Operator "not" Evaluates to "true"
61 Operator "or" Evaluates to "False" (Empty)
62 Operator "or" Evaluates to "False"
63 Operator "or" Evaluates to "true"
64 Operator "or" Evaluates to "true" (All "true")
65 circles-alpha
66 circles-bravo
67 circles-charlie
68 circles-delta
69 circles-echo
70 circles-golf
71 circles-hotel
72 circles-india
73 consistency-throw-hosting-relation-missing
74 consistency-throw-hosting-relation-missing-constraint
75 consistency-throw-multiple-hosting-relations
76 consistency-throw-multiple-hosting-relations-constraint
77 consistency-throw-relation-source-missing
78 consistency-throw-relation-target-missing
79 Operator "equal" Evaluates to "false"
80 Operator "equal" Evaluates to "true"
81 Operator "greater_or_equal" Evaluates to "true" (Greater)
82 Operator "greater_or_equal" Evaluates to "true" (Equal)
83 Operator "greater_or_equal" Evaluates to "false" (Less)
84 Operator "greater" Evaluates to "false" (Equal)
85 Operator "greater" Evaluates to "true" (Greater)
86 Operator "greater" Evaluates to "false" (Less)
87 Operator "in_range" Evaluates to "false"
88 Operator "in_range" Evaluates to "true"
89 Operator "length" Evaluates to "false"
90 Operator "length" Evaluates to "true"
91 Operator "less_or_equal" Evaluates to "true" (Equal)
92 Operator "less_or_equal" Evaluates to "false" (Greater)
93 Operator "less_or_equal" Evaluates to "true" (Less)
94 Operator "less" Evaluates to "false" (Equal)
95 Operator "less" Evaluates to "false" (Greater)
96 Operator "less" Evaluates to "true" (Less)
97 Operator "max_length" Evaluates to "true" (Equal)
98 Operator "max_length" Evaluates to "false" (Greater)
99 Operator "max_length" Evaluates to "true" (Less)
100 Operator "min_length" Evaluates to "true" (Equal)
101 Operator "min_length" Evaluates to "true" (Greater)
102 Operator "min_length" Evaluates to "false" (Less)
103 Operator "valid_values" Evaluates to "false"
104 Operator "valid_values" Evaluates to "true"
105 date-operators-after-false
106 date-operators-after-or-same-false
107 date-operators-after-or-same-true
108 date-operators-after-true
109 date-operators-before-false
110 date-operators-before-or-same-false
111 date-operators-before-or-same-true
112 date-operators-before-true
113 date-operators-same-false
114 date-operators-same-true
115 date-operators-within-false
116 date-operators-within-true
117 Conditional Group
118 groups-default-condition
119 groups-default-condition-nothing
120 groups-group-presence-absent
121 groups-group-presence-present
122 groups-prune
123 hosting-aware-pruning-constraints-elastic
124 hosting-aware-pruning-constraints-static
125 hosting-aware-pruning-fig-10-broken
126 hosting-aware-pruning-fig-10-fixed
127 hosting-aware-pruning-host-elastic
128 hosting-aware-pruning-host-static
129 hosting-aware-pruning-motivation-elastic
130 hosting-aware-pruning-motivation-static
131 hosting-aware-pruning-provider-elastic
132 hosting-aware-pruning-provider-static
133 hosting-aware-pruning-transitive-ambiguous
134 hosting-aware-pruning-transitive-both
135 hosting-aware-pruning-transitive-preserved
136 hosting-aware-pruning-transitive-unexpected
137 implications-constraints
138 implications-implied-left
139 implications-implied-right
140 implications-implies-left
141 implications-implies-right
142 implications-incoming-left
143 implications-incoming-right
144 implications-motivating-scenario-cloud
145 implications-motivating-scenario-static
146 implications-requires
147 implications-unexpected
148 implications-naive-constraints
149 implications-naive-implied-left
150 implications-naive-implied-right
151 implications-naive-implies-left
152 implications-naive-implies-right
153 implications-naive-incoming-left
154 implications-naive-incoming-right
155 implications-naive-motivating-scenario-cloud
156 implications-naive-motivating-scenario-static
157 implications-naive-naive
158 implications-naive-naive-another
159 implications-naive-naive-another-no-optimization
160 implications-naive-naive-another-not
161 implications-naive-naive-not
162 implications-naive-requires
163 implications-naive-unexpected
164 implied-default-fixed-left
165 implied-default-fixed-right
166 implied-default-manual-left
167 implied-default-manual-right
168 implied-host-broken
169 implied-host-fixed-left
170 implied-host-fixed-right
171 implied-host-manual-left
172 implied-host-manual-right
173 import-conditional
174 import-empty
175 import-get-import-absent
176 import-get-import-present
177 inputs-conditional
178 inputs-conditional-list
179 inputs-default-alternative
180 inputs-input-presence-absent
181 inputs-input-presence-present
182 inputs-pruning-consumed
183 inputs-pruning-consumed-still
184 inputs-pruning-consumed-v3
185 inputs-pruning-no-consumer
186 inputs-pruning-not-consumed-anymore
187 inputs-pruning-not-consumed-anymore-v3
188 inputs-unique-input-constraint-sat
189 inputs-unique-input-constraint-sat-other
190 inputs-unique-input-constraint-sat-pruning-consumed
191 inputs-unique-input-constraint-sat-pruning-consumed-default
192 inputs-unique-input-constraint-unsat
193 nodes-conditional
194 nodes-conditional-type
195 nodes-conditional-type-default
196 nodes-default-false-false
197 nodes-default-false-true
198 nodes-default-host-absent
199 nodes-default-host-present
200 nodes-default-true-false
201 nodes-default-true-true
202 nodes-host-presence-absent
203 nodes-host-presence-false-false
204 nodes-host-presence-present
205 nodes-host-presence-true-false
206 nodes-host-presence-true-true
207 nodes-host-presence-true-true-constraint
208 nodes-node-presence-absent
209 nodes-node-presence-present
210 nodes-required-incoming-relation-constraint-sat
211 nodes-required-incoming-relation-constraint-unsat
212 nodes-type-specific
213 Benchmark
214 Operator "concat"
215 Get Variability Expression
216 Operator "concat"
217 other-logic-expression-false
218 other-logic-expression-true
219 other-logic-value-expression
220 other-nothing
221 other-preset
222 other-presets
223 other-throw-CONTAINER
224 other-throw-SELF
225 other-throw-not-unique
226 Operator "concat"
227 Get Variability Expression
228 Get Variability Input
229 other-version
230 other-weight
231 outputs-conditional
232 outputs-conditional-list
233 outputs-default
234 outputs-input-presence-absent
235 outputs-input-presence-present
236 outputs-pruning-not-produced-anymore-unfurl-1
237 outputs-pruning-not-produced-anymore-unfurl-2
238 outputs-pruning-not-produced-anymore-xopera
239 outputs-pruning-not-produced-anymore-xopera-v3
240 outputs-pruning-produced-unfurl
241 outputs-pruning-produced-unfurl-attribute
242 outputs-pruning-produced-unfurl-attribute-not
243 outputs-unique-output-constraint-sat
244 outputs-unique-output-constraint-sat-default
245 outputs-unique-output-constraint-sat-other
246 outputs-unique-output-constraint-sat-pruning-not-produced-anymore-xopera
247 outputs-unique-output-constraint-sat-pruning-produced-xopera
248 outputs-unique-output-constraint-unsat
249 persistent-persistent
250 persistent-persistent-false
251 persistent-persistent-throw
252 persistent-persistent-undefined
253 policies-conditional
254 policies-default-condition
255 policies-default-condition-nothing
256 Policy Targets Absent Members
257 Policy Targets Absent Nodes
258 Policy Has No Targets
259 Policy Targets Present Member
260 Policy Targets One Present Node
261 Policy Targets Present Nodes
262 policies-policy-presence-absent
263 policies-policy-presence-present
264 Policy Pruning
265 properties-artifact-container-presence-absent
266 properties-artifact-container-presence-present
267 properties-conditional
268 properties-default-alternative
269 properties-default-alternative-false
270 properties-default-condition
271 properties-expression
272 properties-list
273 properties-list-another
274 properties-map
275 properties-map-another
276 properties-node-container-presence-absent
277 properties-node-container-presence-present
278 properties-node-node-presence-CONTAINER-absent
279 properties-node-node-presence-CONTAINER-present
280 properties-node-property-presence-number-absent
281 properties-node-property-presence-number-present
282 properties-properties
283 properties-relation-container-presence-absent
284 properties-relation-container-presence-present
285 properties-throw-ambiguous-property
286 properties-throw-missing-property-container
287 properties-throw-multiple-default
288 properties-unique-property-constraint-sat
289 properties-unique-property-constraint-unsat
290 pruning-alpha-default
291 pruning-alpha-default-naive
292 pruning-alpha-pruning
293 pruning-artifact-default-alpha
294 pruning-artifact-default-bravo
295 pruning-artifact-default-charlie
296 pruning-artifact-default-delta
297 pruning-artifact-default-naive
298 pruning-bravo-default
299 pruning-bravo-pruning
300 pruning-charlie-default
301 pruning-charlie-pruning
302 pruning-delta-default
303 pruning-delta-pruning
304 pruning-echo-default
305 pruning-echo-pruning
306 pruning-foxtrott-default
307 pruning-foxtrott-pruning
308 pruning-hotel-default
309 pruning-hotel-pruning
310 pruning-incoming-artifact-pruning-alpha
311 pruning-incoming-artifact-pruning-bravo
312 pruning-incoming-artifact-pruning-charlie
313 pruning-incoming-artifact-pruning-delta
314 pruning-incoming-artifact-pruning-echo
315 pruning-incoming-artifact-pruning-foxtrott
316 pruning-incoming-artifact-pruning-golf
317 pruning-incoming-artifact-pruning-hotel
318 pruning-incoming-artifact-pruning-naive-alpha
319 pruning-incoming-artifact-pruning-naive-bravo
320 pruning-india-default
321 pruning-india-pruning
322 pruning-juliett-default
323 pruning-juliett-pruning
324 pruning-kilo-default
325 pruning-kilo-pruning
326 pruning-lima-default
327 pruning-lima-pruning
328 pruning-mike-default
329 pruning-mike-pruning
330 pruning-outgoing-relation-absent
331 pruning-outgoing-relation-present
332 pruning-outgoing-target-absent
333 pruning-outgoing-target-present
334 pruning-outgoingnaive-target-present
335 pruning-pruning
336 pruning-pruning-false-false
337 pruning-pruning-false-true
338 pruning-pruning-true-false
339 pruning-pruning-true-true
340 Relation Pruning
341 relationships-ambigous-disabled
342 relationships-ambigous-throw
343 relationships-conditional
344 relationships-throw-multiple
345 relationships-throw-undefined
346 relationships-throw-unused
347 relationships-type-specific
348 requirement-assignment-conditional
349 requirement-assignment-default-alternative
350 requirement-assignment-default-alternative-false
351 requirement-assignment-default-condition
352 requirement-assignment-default-condition-throw-source
353 requirement-assignment-default-condition-throw-target
354 requirement-assignment-one-hosting-relation
355 requirement-assignment-relation-presence-index-absent
356 requirement-assignment-relation-presence-name-absent
357 requirement-assignment-source-presence-absent
358 requirement-assignment-source-presence-present
359 requirement-assignment-target-presence-absent
360 requirement-assignment-target-presence-present
361 requirement-assignment-throw-multiple-defaults
362 Scenario "Alpha"
363 scenario-worker-agent-false
364 scenario-worker-agent-true
365 scenario-worker-worker-false
366 scenario-worker-worker-true
367 technologies-ambiguous
368 technologies-ambiguous-disabled
369 technologies-conditional
370 technologies-constraint
371 technologies-constraint-disabled
372 technologies-default-multiple
373 technologies-default-true
374 technologies-expected
375 technologies-expected-disabled
376 technologies-lowercase
377 technologies-missing
378 technologies-missing-disabled
379 technologies-optimization
380 technologies-optimization-double
381 technologies-optimization-order
382 technologies-optimization-unsat
383 technologies-optimization-weight
384 technologies-pruning
385 technologies-pruning-disabled
386 technologies-rules
387 technologies-rules-v3
388 technologies-throw-required
389 technologies-unique
390 technologies-unique-disabled
391 variability-inputs-no-constraints
392 variability-inputs-requires
393 variability-inputs-requires-throw
394 variability-inputs-requires-throw-falsy
395 variability-inputs-requires-throw-truthy

Reference Implementation

We provide a reference implementation for this specification as part of OpenTOSCA Vintner. OpenTOSCA Vintner is a TOSCA preprocessing and management layer. The project is open-source, licensed under Apache-2.0, and publicly available at GitHub.

Acknowledgments

This specification is developed for the purpose of research by the Institute of Software Engineering (ISTE) and the Institute of Architecture of Application Systems (IAAS) of the University of Stuttgart, Germany. The development is partially funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) as part of the Software-Defined Car (SofDCar) project (19S21002).

Correspondence

Please address all correspondence concerning this specification to Miles Stötzner <miles.stoetzner@iste.uni-stuttgart.de, https://miles.stoetzner.de>.

Disclaimer of Warranty

Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.


Last update: November 18, 2024