Skip to content

Variability4TOSCA Conformance Test Suite 1.0 Release Candidate

This document specifies the conformance test suite for the Variability4TOSCA specification which can be used to evaluate Variability4TOSCA implementations. The specification is under active development and is not backwards compatible with any previous versions.

Conformance Tests

Each test is described on a separate page. The description includes typically a variable service template, variability inputs, expected errors, expected variability-resolved service template and so on along with an explanation about the expected processing.

ID Test
1 Operator "count"
2 Operator "exponential_regression"
3 Operator "variance"
4 Operator "logarithmic_regression"
5 Operator "max"
6 Operator "mean"
7 Operator "median"
8 Operator "min"
9 Operator "polynomial_regression"
10 Operator "standard_deviation"
11 Operator "sum"
12 Operator "variance"
13 Operator "add"
14 Operator "div"
15 Operator "mod" (Even)
16 Operator "add" (Odd)
17 Operator "mul"
18 Operator "sub"
19 artifacts-artifact-presence-number-absent
20 artifacts-artifact-presence-number-present
21 artifacts-artifact-presence-string-absent
22 artifacts-artifact-presence-string-present
23 artifacts-conditional
24 artifacts-container-presence-absent
25 artifacts-container-presence-present
26 artifacts-default-alternative
27 artifacts-default-alternative-false
28 artifacts-default-condition
29 artifacts-dont-throw-ambiguous-artifact
30 Artifact Pruning
31 artifacts-pruning-managed-manual
32 artifacts-pruning-managed-rules
33 artifacts-pruning-managed-rules-host
34 artifacts-pruning-managed-rules-multiple-different
35 artifacts-pruning-managed-rules-none
36 artifacts-pruning-manged-rules-multiple-same
37 artifacts-required-artifact-constraint-sat
38 artifacts-required-artifact-constraint-sat-pruning
39 artifacts-required-artifact-constraint-unsat
40 artifacts-throw-ambiguous-artifact
41 artifacts-throw-ambiguous-default
42 artifacts-throw-missing-artifact-container
43 artifacts-unique-artifact-constraint-sat-none
44 artifacts-unique-artifact-constraint-sat-one
45 artifacts-unique-artifact-constraint-sat-other
46 artifacts-unique-artifact-constraint-unsat
47 Operator "and" Evaluates to "True" (Empty)
48 Operator "and" Evaluates to "false"
49 Operator "and" Evaluates to "false" (Nested)
50 Operator "and" Evaluates to "true"
51 Operator "and" Evaluates to "true" (Nested)
52 Operator "implies": false false -> true
53 Operator "implies": false true -> true
54 Operator "implies": true false -> false
55 Operator "implies": true true -> true
56 Operator "not" Evaluates to "false"
57 Operator "not" Evaluates to "true"
58 Operator "or" Evaluates to "False" (Empty)
59 Operator "or" Evaluates to "False"
60 Operator "or" Evaluates to "true"
61 Operator "or" Evaluates to "true" (All "true")
62 Operator "xor": all false -> false
63 Operator "xor": One ture -> true
64 Operator "xor": all true -> false
65 circles-alpha
66 circles-bravo
67 circles-charlie
68 circles-delta
69 circles-echo
70 circles-golf
71 circles-hotel
72 circles-india
73 consistency-throw-hosting-relation-missing
74 consistency-throw-hosting-relation-missing-constraint
75 consistency-throw-multiple-hosting-relations
76 consistency-throw-multiple-hosting-relations-constraint
77 consistency-throw-relation-source-missing
78 consistency-throw-relation-target-missing
79 Operator "equal" Evaluates to "false"
80 Operator "equal" Evaluates to "true"
81 Operator "greater_or_equal" Evaluates to "true" (Greater)
82 Operator "greater_or_equal" Evaluates to "true" (Equal)
83 Operator "greater_or_equal" Evaluates to "false" (Less)
84 Operator "greater" Evaluates to "false" (Equal)
85 Operator "greater" Evaluates to "true" (Greater)
86 Operator "greater" Evaluates to "false" (Less)
87 Operator "in_range" Evaluates to "false"
88 Operator "in_range" Evaluates to "true"
89 Operator "length" Evaluates to "false"
90 Operator "length" Evaluates to "true"
91 Operator "less_or_equal" Evaluates to "true" (Equal)
92 Operator "less_or_equal" Evaluates to "false" (Greater)
93 Operator "less_or_equal" Evaluates to "true" (Less)
94 Operator "less" Evaluates to "false" (Equal)
95 Operator "less" Evaluates to "false" (Greater)
96 Operator "less" Evaluates to "true" (Less)
97 Operator "max_length" Evaluates to "true" (Equal)
98 Operator "max_length" Evaluates to "false" (Greater)
99 Operator "max_length" Evaluates to "true" (Less)
100 Operator "min_length" Evaluates to "true" (Equal)
101 Operator "min_length" Evaluates to "true" (Greater)
102 Operator "min_length" Evaluates to "false" (Less)
103 Operator "valid_values" Evaluates to "false"
104 Operator "valid_values" Evaluates to "true"
105 date-operators-after-false
106 date-operators-after-or-same-false
107 date-operators-after-or-same-true
108 date-operators-after-true
109 date-operators-before-false
110 date-operators-before-or-same-false
111 date-operators-before-or-same-true
112 date-operators-before-true
113 date-operators-same-false
114 date-operators-same-true
115 date-operators-within-false
116 date-operators-within-true
117 Conditional Group
118 groups-default-condition
119 groups-default-condition-nothing
120 groups-group-presence-absent
121 groups-group-presence-present
122 groups-prune
123 hosting-aware-pruning-constraints-elastic
124 hosting-aware-pruning-constraints-static
125 hosting-aware-pruning-fig-10-broken
126 hosting-aware-pruning-fig-10-fixed
127 hosting-aware-pruning-host-elastic
128 hosting-aware-pruning-host-static
129 hosting-aware-pruning-motivation-elastic
130 hosting-aware-pruning-motivation-static
131 hosting-aware-pruning-provider-elastic
132 hosting-aware-pruning-provider-static
133 hosting-aware-pruning-transitive-ambiguous
134 hosting-aware-pruning-transitive-both
135 hosting-aware-pruning-transitive-preserved
136 hosting-aware-pruning-transitive-unexpected
137 implications-constraints
138 implications-implied-left
139 implications-implied-right
140 implications-implies-left
141 implications-implies-right
142 implications-incoming-left
143 implications-incoming-right
144 implications-motivating-scenario-cloud
145 implications-motivating-scenario-static
146 implications-requires
147 implications-unexpected
148 implications-naive-constraints
149 implications-naive-implied-left
150 implications-naive-implied-right
151 implications-naive-implies-left
152 implications-naive-implies-right
153 implications-naive-incoming-left
154 implications-naive-incoming-right
155 implications-naive-motivating-scenario-cloud
156 implications-naive-motivating-scenario-static
157 implications-naive-naive
158 implications-naive-naive-another
159 implications-naive-naive-another-no-optimization
160 implications-naive-naive-another-not
161 implications-naive-naive-not
162 implications-naive-requires
163 implications-naive-unexpected
164 implied-default-fixed-left
165 implied-default-fixed-right
166 implied-default-manual-left
167 implied-default-manual-right
168 implied-host-broken
169 implied-host-fixed-left
170 implied-host-fixed-right
171 implied-host-manual-left
172 implied-host-manual-right
173 import-conditional
174 import-empty
175 import-get-import-absent
176 import-get-import-present
177 inputs-conditional
178 inputs-conditional-list
179 inputs-default-alternative
180 inputs-input-presence-absent
181 inputs-input-presence-present
182 inputs-pruning-consumed
183 inputs-pruning-consumed-still
184 inputs-pruning-no-consumer
185 inputs-pruning-not-consumed-anymore
186 inputs-pruning-not-consumed-anymore-v3
187 inputs-unique-input-constraint-sat
188 inputs-unique-input-constraint-sat-other
189 inputs-unique-input-constraint-sat-pruning-consumed
190 inputs-unique-input-constraint-sat-pruning-consumed-default
191 inputs-unique-input-constraint-unsat
192 nodes-conditional
193 nodes-conditional-type
194 nodes-conditional-type-default
195 nodes-default-false-false
196 nodes-default-false-true
197 nodes-default-host-absent
198 nodes-default-host-present
199 nodes-default-true-false
200 nodes-default-true-true
201 nodes-host-presence-absent
202 nodes-host-presence-false-false
203 nodes-host-presence-present
204 nodes-host-presence-true-false
205 nodes-host-presence-true-true
206 nodes-host-presence-true-true-constraint
207 nodes-node-presence-absent
208 nodes-node-presence-present
209 nodes-required-incoming-relation-constraint-sat
210 nodes-required-incoming-relation-constraint-unsat
211 nodes-type-specific
212 Benchmark
213 Operator "concat"
214 Get Variability Expression
215 Operator "concat"
216 other-logic-expression-false
217 other-logic-expression-true
218 other-logic-value-expression
219 other-nothing
220 other-preset
221 other-presets
222 other-throw-CONTAINER
223 other-throw-SELF
224 other-throw-not-unique
225 Operator "concat"
226 Get Variability Expression
227 Get Variability Input
228 other-version
229 other-weight
230 outputs-conditional
231 outputs-conditional-list
232 outputs-default
233 outputs-input-presence-absent
234 outputs-input-presence-present
235 outputs-pruning-not-produced-anymore-unfurl-1
236 outputs-pruning-not-produced-anymore-unfurl-2
237 outputs-pruning-not-produced-anymore-xopera
238 outputs-pruning-not-produced-anymore-xopera-v3
239 outputs-pruning-produced-unfurl
240 outputs-pruning-produced-unfurl-attribute
241 outputs-pruning-produced-unfurl-attribute-not
242 outputs-unique-output-constraint-sat
243 outputs-unique-output-constraint-sat-default
244 outputs-unique-output-constraint-sat-other
245 outputs-unique-output-constraint-sat-pruning-not-produced-anymore-xopera
246 outputs-unique-output-constraint-sat-pruning-produced-xopera
247 outputs-unique-output-constraint-unsat
248 persistent-persistent
249 persistent-persistent-false
250 persistent-persistent-throw
251 persistent-persistent-undefined
252 policies-conditional
253 policies-default-condition
254 policies-default-condition-nothing
255 Policy Targets Absent Members
256 Policy Targets Absent Nodes
257 Policy Has No Targets
258 Policy Targets Present Member
259 Policy Targets One Present Node
260 Policy Targets Present Nodes
261 policies-policy-presence-absent
262 policies-policy-presence-present
263 Policy Pruning
264 properties-artifact-container-presence-absent
265 properties-artifact-container-presence-present
266 properties-conditional
267 properties-default-alternative
268 properties-default-alternative-false
269 properties-default-condition
270 properties-expression
271 properties-list
272 properties-list-another
273 properties-map
274 properties-map-another
275 properties-node-container-presence-absent
276 properties-node-container-presence-present
277 properties-node-node-presence-CONTAINER-absent
278 properties-node-node-presence-CONTAINER-present
279 properties-node-property-presence-number-absent
280 properties-node-property-presence-number-present
281 properties-node-property-presence-string-absent
282 properties-node-property-presence-string-present
283 properties-properties
284 properties-relation-container-presence-absent
285 properties-relation-container-presence-present
286 properties-throw-ambiguous-property
287 properties-throw-missing-property-container
288 properties-throw-multiple-default
289 properties-unique-property-constraint-sat
290 properties-unique-property-constraint-unsat
291 pruning-alpha-default
292 pruning-alpha-default-naive
293 pruning-alpha-pruning
294 pruning-artifact-default-alpha
295 pruning-artifact-default-bravo
296 pruning-artifact-default-charlie
297 pruning-artifact-default-delta
298 pruning-artifact-default-naive
299 pruning-bravo-default
300 pruning-bravo-pruning
301 pruning-charlie-default
302 pruning-charlie-pruning
303 pruning-delta-default
304 pruning-delta-pruning
305 pruning-echo-default
306 pruning-echo-pruning
307 pruning-foxtrott-default
308 pruning-foxtrott-pruning
309 pruning-hotel-default
310 pruning-hotel-pruning
311 pruning-incoming-artifact-pruning-alpha
312 pruning-incoming-artifact-pruning-bravo
313 pruning-incoming-artifact-pruning-charlie
314 pruning-incoming-artifact-pruning-delta
315 pruning-incoming-artifact-pruning-echo
316 pruning-incoming-artifact-pruning-foxtrott
317 pruning-incoming-artifact-pruning-golf
318 pruning-incoming-artifact-pruning-hotel
319 pruning-incoming-artifact-pruning-naive-alpha
320 pruning-incoming-artifact-pruning-naive-bravo
321 pruning-india-default
322 pruning-india-pruning
323 pruning-juliett-default
324 pruning-juliett-pruning
325 pruning-kilo-default
326 pruning-kilo-pruning
327 pruning-lima-default
328 pruning-lima-pruning
329 pruning-mike-default
330 pruning-mike-pruning
331 pruning-outgoing-relation-absent
332 pruning-outgoing-relation-present
333 pruning-outgoing-target-absent
334 pruning-outgoing-target-present
335 pruning-outgoingnaive-target-present
336 pruning-pruning
337 pruning-pruning-false-false
338 pruning-pruning-false-true
339 pruning-pruning-true-false
340 pruning-pruning-true-true
341 Relation Pruning
342 relationships-conditional
343 relationships-throw-multiple
344 relationships-throw-undefined
345 relationships-throw-unused
346 relationships-type-specific
347 requirement-assignment-conditional
348 requirement-assignment-default-alternative
349 requirement-assignment-default-alternative-false
350 requirement-assignment-default-condition
351 requirement-assignment-default-condition-throw-source
352 requirement-assignment-default-condition-throw-target
353 requirement-assignment-one-hosting-relation
354 requirement-assignment-relation-presence-index-absent
355 requirement-assignment-relation-presence-name-absent
356 requirement-assignment-source-presence-absent
357 requirement-assignment-source-presence-present
358 requirement-assignment-target-presence-absent
359 requirement-assignment-target-presence-present
360 requirement-assignment-throw-multiple-defaults
361 Scenario "Alpha"
362 scenario-worker-agent-false
363 scenario-worker-agent-true
364 scenario-worker-worker-false
365 scenario-worker-worker-true
366 technologies-ambiguous
367 technologies-ambiguous-disabled
368 technologies-conditional
369 technologies-constraint
370 technologies-constraint-disabled
371 technologies-default-multiple
372 technologies-default-true
373 technologies-expected
374 technologies-expected-disabled
375 technologies-lowercase
376 technologies-missing
377 technologies-missing-disabled
378 technologies-optimization
379 technologies-optimization-double
380 technologies-optimization-order
381 technologies-optimization-unsat
382 technologies-optimization-weight
383 technologies-pruning
384 technologies-pruning-disabled
385 technologies-rules
386 technologies-rules-v3
387 technologies-throw-required
388 technologies-unique
389 technologies-unique-disabled
390 variability-inputs-implies
391 variability-inputs-no-constraints

Reference Implementation

We provide a reference implementation for this specification as part of OpenTOSCA Vintner. OpenTOSCA Vintner is a TOSCA preprocessing and management layer. The project is open-source, licensed under Apache-2.0, and publicly available at GitHub.

Acknowledgments

This specification is developed for the purpose of research by the Institute of Software Engineering (ISTE) and the Institute of Architecture of Application Systems (IAAS) of the University of Stuttgart, Germany. The development is partially funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) as part of the Software-Defined Car (SofDCar) project (19S21002).

Correspondence

Please address all correspondence concerning this specification to Miles Stötzner <miles.stoetzner@iste.uni-stuttgart.de, https://miles.stoetzner.de>.

Disclaimer of Warranty

Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.


Last update: September 5, 2024