Using Tiers of Assurance Evidence to Reduce the Tears! Adopting the “Wheel of Qualification” for an Alternative Software Safety Assurance Approach

Mark Hadley

Senior Principal Consultant - Software and Systems, DSTL

Mike Standish

Senior Engineer - Systems, DSTL

A traditional method to gain confidence in software is to develop it using a life-cycle centred process, which is measured against a set of predefined objectives. A judgement on the level of compliance to these objectives is taken which allows a degree of confidence in the software to be stated. However, if only certain types of evidence are accepted to demonstrate compliance, e.g. process-based evidence, then the scope of the systems considered are reduced, or the system may be subject to operating limitations.


Any system must still be supported by evidence that demonstrates the safety requirements are met. The use of diverse evidence can achieve an equivalent level of compliance to a full process-based approach. Therefore diverse evidence can form part of a software safety assurance strategy. For a number of systems there will be additional confidence building activities conducted by Design Organisations (DOs), Coordinating DOs (CDOs), and Independent Technical Evaluators, for example. These activities go beyond the standard life-cycle review. Dstl have devised (and are currently implementing) an alternative approach which allows for wider, more diverse, activities to be used in assuring a system’s underpinning software and Complex Electronic Hardware (CEH). Diverse evidence can be challenging to measure when compared to traditional process-based approaches; however, Dstl has devised solutions to ameliorate such difficulties via methods such as a stakeholder communication model.

About Mark Hadley

Mark has been involved in the airborne safety critical software domain for over 20 years with the UK Defence Science and Technology Laboratory (Dstl) (and its predecessor organisations) working on a range of UK Ministry of Defence (MOD) airborne systems. Mark is a senior principal consultant in software and provides Independent Technical Evaluation (ITE) and Subject Matter Expert (SME) advice to a host of MOD Project Teams. He is leading research into a number of areas such as: multi-core processors and the generation of diversity of evidence arguments to support the qualification of mission and safety critical systems. Mark completed his PhD in software testing at the University of York. He is a Chartered Engineer (CEng) gained via the Institution of Engineering and Technology (IET).

About Mike Standish

Mike is a senior engineer in systems at the UK Defence Science and Technology Laboratory (Dstl). Mike has experience of all aspects of software and systems lifecycles, which has been gained in over 15 years within the defence sector. Mike holds a BSc in Software Engineering and an MSc in Strategic Information Systems. Mike is currently undertaking an Engineering Doctorate (EngD) in Systems at the University of Bristol with a focus on how to adopt wider diverse evidence to mitigate shortfalls in software process-based safety assurance evidence. He is a Chartered Engineer (CEng) gained via the British Computer Society (BCS).

Sponsored by