and I find it very helpful when you have to deal with non trivial systems, you fear to miss something of important or there is a combinatorial explosion.
ACTS let you create a model for your system defining parameters, boundary values, different types of inputs, valid and invalid data, relations between parameters and different generation algorithms, base choices (e.g., more important values you want to be used most often).
More information from the ACTS used guide:
ACTS is a test generation tool for constructing t-way combinatorial test sets.Always from the ACTS user guide:
Currently, it supports t-way test set generation with t ranging from 1 to 6.
Combinatorial testing has been shown very effective in detecting faults that are caused by unexpected interactions between different contributing factors.
A system is specified by a set of parameters and their values. A test set is a t-way test set if it satisfies the following property: Given any t parameters (out of all the parameters) of a system, every combination of values of these t parameters is covered in at least one test in the test set.So ACTS helps you to drastically reduce the testing combinations for moderate or very large systems. The most important ACTS output is the test set generation (e.g., a calc/Excel export file). The coverage function from the statistics tab is interesting too.
Currently, ACTS supports t-way test set generation for 1 = t = 6. Empirical studies show that t being up to 6 is sufficient for most practical applications. A special form of 1-way testing, called base-choice testing, is implemented in ACTS. Base-choice testing requires that every parameter value be covered at least once and in a test in which all the other values are base choices. Each parameter has one or more values designated as base choices. Informally, base choices are "more important" values, e.g., default values, or values that are used most often in operation.
Since we have finite time and infinite test combinations, we have to be very
selective and ACTS is a valid help.
I find that the ACTS test cases export sometimes is not readable and you loose the base choice info, invalid data and you might be lost in combinations. I think it is a good idea adding some simple formulas and conditional formatting for improving the readability of the export, see base choices, distinguish between positive and negative scenarios (e.g., yellow), emphasize when you get the same output from different paths, visualize different shapes of test cases and convey more meaningful information about your data depending on your domain.
This way you will be able to detect more easily what are the most important cases and prioritize them (e.g., feature confirmation and guessing the test cases with a higher probability of failure) for both positive and negative cases to be as effective as possible even if you have very little time for testing.
Obviously before creating the ACTS project you should already have an idea of what are the most important scenarios and all the possible ways for obtaining a certain result but thanks to ACTS you can have a visual double check and see if you missed something of important or other edge cases.
When you identify the most interesting rows you can add a description note explaining why this particular scenario is important, prioritize all your test cases and add all the needed information and adding new scenarios or edge cases. The file is ready to be imported to a test management tool now and you have also an ordered execution plan based on priority and risk.
And if you don't have enough time for testing the exported combinations you might evaluate if you have sufficient information to realize if the most relevant cases are covered, the level of confidence, coverage and you can evaluate if the residual risk is acceptable.
This test set could be also used for automating your scenarios. If you use for example a keyword driven testing pattern or more abstract patterns you can implement just one time the needed logics and with little time you will cover all your hundreds of combinations because your test framework let you decouple the test data from the test implementation. In following articles we'll discuss about pytest fixtures/parametrization, pytest-bdd and other test frameworks. Stay tuned!
Links:
No comments:
Post a Comment
Note: only a member of this blog may post a comment.