Why does so much IT fail? — and what to do about it
Your first reaction to this title might be that your iPhone and iPad are wonderful (and Google Glasses are even better…), so if only we brought everything up to date all would be well. Indeed, we may think IT is wonderful, but the reality includes a wide variety of failures: from the heartbleed bug, international programs to computerise healthcare, and an embarrassing array of preventable disasters.
We start with the high-profile failure of the German World War II Enigma, the design weaknesses of which undeniably helped lead to the war's conclusion — the German cryptographers, despite high motivation, failed to spot design flaws in the technology. In hindsight it is now easy to see the reasons, but it provides a fascinating object lesson.
The same symptoms present everywhere today in all IT systems. Interactive behaviour is very hard to understand.
As the examples in this talk come up to date, they get closer to our own stakeholding, and we get more resistant to the issues.
Medical devices with embedded IT suffer from ubiquitous preventable design problems, but they harm people. In a world where preventable death in hospitals is the third major killer (in the UK, killing about 90,000 per year — just behind heart disease and lung cancer — eclipsing, by a factor of 40, road fatalities), medical IT is making a worrying contribution. An ubiquitous but completely avoidable problem is bad number entry design, where user errors (which one expects and should design for) have unpredictable effects on the devices.
We need rigorous approaches to user interface design and evaluation for safety critical applications, packaged as tools so they are easy and reliable to use. We have built stochastic tools and tools on top of PVS (a theorem prover) to help identify problems that are otherwise overlooked. Improved design is easy once you decide to do it, and we have shown that significant risk reduction is possible. The bigger issue is to persuade the world.