Expert testing without the experts
Thomas Wilson
Senior Architect for Test, High-Integrity Expertise Centre, Capgemini Engineering
Techniques for testing high-integrity systems are widely-known within industry, but the standard techniques are highly effort-intensive. Within Capgemini, like many organisations, we are striving to produce the most effective test approach possible for high-integrity systems, which minimises costs while retaining the required levels of quality. In this talk, I will outline our approach to this problem. We utilise model-based specifications that we can then process using tools that automate the skilled task of test specification and design. We have found that the systematic approaches that our test teams have been employing for decades were greatly amenable to automation once we selected the right modelling language and fully utilised the supporting tools. What results is an innovative approach that can automate a high proportion of the functional high-integrity system testing in a qualifiable way.
Our approach has been developed over the last 5 years under the SECT-AIR and HICLASS research projects. We use an algorithm to generate tests from the Alloy modelling language. The algorithm analyses the specification model and applies equivalence partition and boundary value analysis to produce a set of Verification Conditions that capture the required specification coverage. State-of-the-art solvers are then used to generate test data that satisfies these conditions. Implementations in any programming language can be tested against the model, using one of the adapters provided. Specialised adapters are provided for testing Ada and Java, which minimise the required test harness work, but other languages can be tested by writing a harness in that language that parses our JSON data format.
I will describe our general test approach, illustrated through demonstration of it being applied in practice, and share insights into how to apply an approach like this successfully. We have produced one realisation of this general approach in a tool called Crucible, but we will explain how the approach could be applied to other modelling and implementation languages.
About Thomas Wilson
Thomas Wilson bio: Dr Thomas Wilson is Senior Architect for Test at the High-Integrity Expertise Centre of Capgemini Engineering, based in Bath. He has worked for Capgemini Engineering for 16 years, split between doing high-integrity systems engineering and carrying out research to make that engineering more efficient. He has experience across all phases of the engineering process and across a range of sectors (particularly aerospace, defence and rail). He has a Ph.D. in Applied Formal Methods and is an expert in the practical application of formal techniques within industrial software engineering. For the last 4 years, he has acted as Technical Authority for Capgemini Engineering on the HICLASS research project, overseeing the development of, amongst other things, tools for automating the production of tests for high-integrity systems. In his role as Senior Architect for Test for the High-Integrity Expertise Centre of Capgemini Engineering, he is now working with a broader remit to establish a next generation high-integrity test approach for Capgemini Engineering.
Sponsored by
