The software industry is in crisis. A strong claim? The National Institute of Standards and Technology (NIST) reports that poor quality software costs the US economy $60 Billion per year [1]. According to the aptly named Chaos report only a quarter of software projects are judged a success [2]. Failures due to "computer glitches" are commonplace, and seem to be viewed by the public (if not by the software industry itself) as inevitable. In any other engineering discipline, or indeed any field, engineering or otherwise, this would be unacceptable. But in the safety and security sector, where reliance on correctly functioning software is increasing, and where such software is becoming ever larger and more complex, this state of affairs is unsustainable.

The challenge for the software industry has been neatly summarized by Martyn Thomas, visiting Professor of Software Engineering at Oxford University in England, as follows: "The only way to reduce costs and to keep projects within plans is to dramatically reduce the error rate at every stage in the development. If you do that, the product is not only cheaper, but higher quality: more secure, more reliable, and easier to maintain."

Thomas's emphasis on reducing errors has been backed up by recent work on behalf of the National Cyber Security Partnership, formed in 2003 in response to the White House National Strategy to Secure Cyberspace [3]. The Partnership's Secure Software Task Force reported that a primary cause of security problems is software with vulnerabilities caused by defects, or errors in software [4], and that practices which lead to low defect software are therefore to be encouraged.

One such practice cited in the report, an approach known as Correctness by Construction developed by Praxis High Integrity Systems Limited in England, has been used for over fifteen years to produce very low defect software in critical applications. As well as being low defect, such software has also proved to be highly cost-effective to develop and maintain in operation. Two examples are cited below, including a zero defect security application.

However, given that Correctness by Construction (and the other best practice approaches cited in the report) has been used successfully for a number of years, some questions arise. Why is there still so much poor quality software around? Why are these approaches not in more widespread use? Perhaps the real challenge for the software industry is to find a way of systematically applying known technology?

Before exploring these questions, it is worth summarizing the Correctness by Construction software development approach, and some examples of its results. The underlying principles are straightforward: firstly to make it difficult to introduce errors, and secondly to remove any errors as soon as possible after introduction. The key to achieving this is to introduce sufficient precision at each step of the development of the software to enable reasoning about the correctness of that step. The correctness of the software can then be demonstrated in terms of the manner in which it has been produced (the "by construction") rather than just by observing operational behavior. An analogy may be drawn with aeronautical engineering, where the demonstration of correctness during the specification, design and implementation phase is such that it is rare for a new aircraft to work incorrectly the first time operational behavior is observed!

It is the use of precision which differentiates Correctness by Construction from other approaches. While perhaps relying only on good process, many other software development approaches endure a lack of precision which makes it very easy to introduce errors, and very hard to find those errors early. Evidence for this may be found in the common tendency for development lifecycles to migrate to an often-repeating "code-test-debug" phase, which can lead to severe cost and timescale overruns. So, what kind of precision does Correctness by Construction use?

At the requirements step (a source of half of project failures [2]) a clear distinction is made between user requirements, system specifications and domain knowledge, and "satisfaction arguments" are used to show that each user requirement can be satisfied by an appropriate combination of system specification and domain knowledge. The emphasis on domain knowledge is key; half of all requirements errors are related to domain [5], yet the vast majority of requirements processes do not explicitly address issues in the domain.

At the specification and design stages, mathematical (or formal) methods and notations are used to define precisely the behavior of the software, and to model its characteristics (for example demonstrating that a multi-process design cannot deadlock). Such techniques can allow precise verification of consistency and accuracy.

At the detailed design and implementation stages, information and data flows are explicitly modeled and statically analyzed (for example, to demonstrate the separation of secure state). Where applicable, the code is written in a mathematically verifiable programming language and statically analyzed (for example, to demonstrate the absence of run-time errors, such as buffer overflows, which are the bane of secure systems).

Correctness by Construction is cost-effective because errors are eliminated early or not introduced in the first place, dramatically reducing the amount of rework needed later in the development. The precision means that the requirements are more likely to be correct, and the system more likely to be the correct system to meet the requirements, and to work correctly in operation. Software developed using Correctness by Construction has also proved to be very cost-effective to maintain.

The results speak for themselves. Correctness by Construction was used by Praxis to develop the Certification Authority system to support the MULTOS multi-application smart card operating system developed by Mondex International (now part of Mastercard) [6]. Developed to the standards of the IT Security Evaluation Criteria (ITSEC) [7] Level E6 (roughly equivalent to Common Criteria [7] Evaluation Assurance Level (EAL) 7), the system had to meet both stringent security requirements and demanding availability requirements. The system was delivered with a warranty against defects, and had an operational defect rate of 0.04 defects/kloc (thousand lines of code), yet was developed at a productivity of almost 30 loc/day (three times typical industry figures).

In the US, Praxis used Correctness by Construction to develop a demonstrator biometrics system for the National Security Agency (NSA), aimed at showing that it is possible to produce high-quality, low defect software conforming to the Common Criteria [6] requirements for Evaluation Assurance Level (EAL) 5 and above [8]. The software was subjected to rigorous independent reliability testing which identified zero defects, and was developed at a productivity of well over 30 loc/day.

More generally, Correctness by Construction delivers software with defect rates of 0.1 defects/kloc and lower; this compares very favorably with defect rates reported by Capability Maturity Model (CMM) Level 5 organizations of 1 defect/kloc [9] (see chart Figure 1).

Figure 1

So, given the apparent success of best practice approaches such as Correctness by Construction, why is there still so much poor quality software around, and why is such best practice not in more widespread use?

There seem to be two kinds of barriers to the adoption of best practice. Firstly, there is often a cultural mindset or awareness barrier. Many individuals and organizations do not, or do not want to, recognize or believe that it is possible to develop software that is low defect, secure and cost-effective. This may simply be an awareness issue, in principle readily addressed by articles such as this, or papers such as those cited. Or there may be a view that such best practice "could never work here" for a combination of reasons. These reasons are likely to include perceived capability of the in-house staff, belief about applicability to the organization's product or product development approach, prevalence of legacy software which is viewed as inherently difficult and therefore expensive to maintain, or concern about the disruption and cost of introducing new approaches.

Secondly, where the need for improvement is acknowledged and considered achievable there are usually practical barriers to overcome, such as how to acquire the necessary capability or expertise, and how to introduce the changes necessary to make the improvements. Introducing such change may be challenging for a combination of technical, political and social reasons.

These are reasonable, common, but not insurmountable barriers, and overcoming them requires effort from suppliers, procurers and regulators and involvement at the individual, project and organizational level. Typically, strong motivation and leadership will be required at a senior management level, where the costs to the business of poor quality (high defects, low productivity) are most likely to be experienced.

At a supplier level, a typical way forward is for organizations (and individuals within them) to take a fresh, open-minded look at what is possible by comparing current approaches to best practice and, where appropriate, adopting step-wise, prioritized improvements based on assessments of the Return On Investment. Engineering decisions on process, methods and tools need to be premised on the basis of logic and precision (for example by asking "how does this choice help me reason about my software?"), rather than on silver bullets or fashion (characterized by questions such as "how many developers already know this particular technology?").

Procurers and regulators can help by adopting an attitude of not settling for less, by demanding a warranty, by awarding contracts to organizations with the capability to deliver low-defect software, and by using contracting arrangements such as gain-share that encourage partnership and improvement.

Fundamentally, however, the main drivers for change will come from two directions.

Regulation, such as in the form of the Common Criteria [7], at least at EAL 5 and above, already requires the adoption of techniques that provide demonstration of correctness through the way software is developed. As reliance on correctly functioning software-intensive security applications increases, and where such software is becoming ever larger and more complex, the prevalence of requirements for EAL 5 and above will increase, and the software industry will need to adapt its development approaches in order to meet these requirements. The situation is analogous to the safety-critical sector, particularly in Europe, where the key safety regulatory requirements now require such approaches. This is the "stick" incentive, and there is a view that if the industry persists in producing insecure software then regulation will increase.

But there is also the "carrot" incentive. There is plenty of evidence from a range of sectors that shows that best practice software engineering produces high quality software cost-effectively. When organizations recognize that low defect software really can have through-life cost benefit (even taking into account the costs of the time and effort to acquire the capability to deliver it) then the business driver will take over Đ the $60 billion reported by NIST [1] is a big prize!

Perhaps the real challenge for the software industry is to recognize and eat the "carrot" before being beaten by the "stick".

 

About the author

Martin is Associate Director for security with Praxis High Integrity Systems Limited, a UK-based systems engineering company specializing in mission-critical systems. Martin Croxford is a chartered engineer with 15 years experience in the software industry. Martin has worked on software development projects in a range of organizations, and as a software development manager has used Correctness by Construction to successfully deliver a multi-million pound security-critical system.

 

Contact Details

Praxis High Integrity Systems Limited

20 Manvers Street, Bath BA1 1PX, UK

http://www.praxis-his.com/

+44 1225 466991 (switchboard)

+44 7881 516750 (cell)

[email protected]

 

References

1  US NIST Report 7007.011, May 2002

2  The Chaos Report http://www.standishgroup.com

3  http://www.cyberpartnership.org/about-overview.html

4  Processes to Produce Secure Software http://www.cyberpartnership.org/SDLCFULL.pdf

5  Hooks and Farry, Customer Centred Products, Amacom, 2000

6  Correctness by Construction: Developing a Commercial Secure System, Anthony Hall and Roderick Chapman. Published in IEEE Software January/February 2002 pp 18-25. http://www.praxis-his.com/pdfs/c_by_c_secure_system.pdf

7  Information about ITSEC and the Common Criteria may be referenced from http://www.cesg.gov.uk/site/iacs/index.cfm

8  Fourth Annual High Confidence Software and Systems Conference Proceedings, National Security Agency, April 13-15 2004

9  Jones, Capers: Software Assessments, Benchmarks, and Best Practices. Reading, MA: Addison-Wesley, 2000