Team Software Process Reliability Results

John B. Goodenough, Software Engineering Institute (SEI)

Introduction

The Team Software Process (TSP)¹ defines how to create and manage software development teams. Software engineers who will become team members are first trained in how to do quality work, follow a defined process, and make process measurements that improve the quality of their work. This training is provided by an intensive 120-hour course that teaches the Personal Software Process^SM(PSP)². The PSP course is very effective in teaching software engineers how to develop schedules they can meet, how to monitor their progress, and how to detect and remove defects in their code. The discipline followed by PSP-trained engineers enables high performance when working in teams, but individual process discipline is not enough. The Team Software Process provides the additional guidance individual engineers need to work effectively as part of teams.

With the TSP, each project starts with a team launch guided by TSP forms, scripts, and standards. The launch process is not a training exercise; team members work directly on their project, establishing team roles, team goals, detailed task plans (typically ten task-hours or less for the lowest level tasks), risks, assignments to tasks, and a presentation of the project's plan to management. The team meets weekly to measure progress and make any needed adjustments. Since the team has developed team goals and implementation plans, its members strive to meet the team's objectives.

The Team Software Process (TSP)

The Team Software Process has been developed by the Software Engineering Institute. The results presented here are drawn from 18 projects reported by four organizations: Boeing, Hill AFB, AIS (a small software company), and Teradyne. Data from the projects have been combined to illustrate team performance on several dimensions, before and after using TSP. Before/after ranges are shown for deviation from schedule, system test duration, acceptance test defects/KLOC, and post-release defects/KLOC. ("Before" data was not reported for every dimension by every organization.)

The effect of TSP on deviation from predicted schedule is striking. The organizations reporting schedule deviation data prior to their use of TSP said their average deviation from planned schedules ranged from 27% for the best reporting organization to 112% for the organization reporting the worst performance. For the 18 projects using TSP, the deviation from planned schedules ranged from -8% to +5%. The results are shown in Figure 1. This reflects a dramatic improvement in achieving planned schedules compared to the previous experience of the reporting organizations.

PSP training dramatically increases the ability of software engineers to detect defects injected in code prior to system test. Using the Team Software Process reinforces and encourages continued use of PSP techniques by all team members. Because defects are detected early and thoroughly, system test durations are drastically shortened, and the number of defects detected at acceptance test and after delivery are extremely low. As shown in Figure 2, system test duration decreased from an average of one to five days per KLOC to 0.1 to 1.0 days/KLOC. The number of acceptance test defects reported prior to use of TSP ranged from 0.1 to 0.7 defects per KLOC (see Figure 3). (The organization reporting 0.1 defects/KLOC was a CMM" Level 5 organization.) Acceptance test defects with the use of TSP ranged from 0.02 defects/KLOC (for the Level 5 organization) to 0.1 defects/KLOC (for all the other organizations, including the organizations operating at CMM Level 2). And finally, reported post-delivery defects (see Figure 4) ranged from zero to 0.1 defects/KLOC, a dramatic difference from the previous performance of these organizations.

In short, the investment in PSP and TSP training pays off in dramatically improved quality and dramatically improved ability to meet schedules, with improvements being exhibited for both low and high maturity organizations.

About the Author

Author Contact Information

John Goodenough is the Chief Technical Officer of the SEI and was named a fellow of the Association for Computing Machinery (ACM) in 1995. He is the former leader of the Rate Monotonic Analysis for Real-Time Systems Project. He was a Distinguished Reviewer for the Ada 95 language revision effort and has served as head of the U.S. delegation to the ISO Working Group on Ada. He was the principal author of the document specifying the revision requirements for Ada 95 and has served as chair of the group responsible for recommending interpretations of the Ada language. Before joining the SEI, Goodenough was manager of the research and development department of SofTech, Inc. His work focused on the Ada programming language. He was the principal designer of one of the candidate languages leading to Ada. He later supported the Ada development effort as a distinguished reviewer for the Department of Defense, led the Ada Compiler Validation effort, and helped develop Ada training materials. Goodenough has worked at the Wang Institute of Graduate Studies as a visiting scholar, where he lectured on software reusability and testing and led seminars on object-oriented languages. He also has worked at the Air Force Electronic Systems Division in Bedford, Mass. There, he was responsible for formulating contract and in-house research and development, and he sponsored the first research work on software maintenance.

John B. Goodenough Software Engineering Institute Carnegie Mellon Pittsburgh, PA 15213-3890 Phone: (412) 268-6391 Fax: (412) 268-5758 [email protected] www.sei.cmu.edu/staff/jbg/ Results from 18 projects executed by several organizations at different levels of process maturity show that use of the Team Software Process (TSP)SM yields dramatic improvement in the ability of software engineers to produce very high quality software, on time, at the predicted cost. A detailed presentation of these results is to be given at the Software Technology Conference in May 2000.

---