The original DARPA STARS project had a number of objectives as it sought to infuse leading edge software engineering technology into the Department of Defense (DoD). One of those objectives was to develop well-defined processes that were not only repeatable, but practical and effective in developing software that was better, faster and cheaper. The Air Force STARS/SCAI demonstration project developed a process that combined aspects of the Cleanroom methodology, the Ada Process Model and an object-oriented, state based approach. As the AF STARS demonstration project was winding down, a critical, short-fused requirement surfaced that needed to provide an automated capability to NORAD system controllers in Cheyenne Mountain. This new development project, called ATAMS (Automated Tracking and Monitoring System), was to be fielded in several phases, but the first fully operational phase was to be deployed within one year. Because of the success of the STARS/SCAI demonstration project, the Air Force decided to waive its standard approach and use the SCAI process. ATAMS was deployed early this year, on-time, on budget, and with a user that is delighted with the system. What follows is a very brief overview of some of the major aspects of the State-Based Software Development Process that was used throughout the ATAMS project.

The State-Based Process: The process was enacted within the framework of an evolutionary, incremental approach derived from the Ada Process Model and the Cleanroom methodology. Our overall approach was to develop the software following an overall incremental approach where each increment would be fully executable, testable, and demonstrable to the user for observation and approval. The Application Engineering (AE) team believed that a software development process definition could be developed and described using an Object Oriented (OO) approach. This primarily led to viewing a particular development item as an object undergoing a series of transitions from one state to another with state operations that related to both the OO and Cleanroom methodologies.

Some of the objectives that drove our approach to this process definition were to tie the management process directly to the technical process; fuse OO and Cleanroom methodologies; maximize the use of inspections and state-of-the-art process methodology; and emphasize simplicity in all aspects of the definition and enactment. The resultant process uses a state transition diagram to describe the states through which a particular object must transition to reach its final state. Figure 1. depicts the overall view of the State-Based Development Process.

A Process Guide was developed to detail the process and management activities that must occur within each state and each transition. Team walkthroughs constitute the major activity in which the state of the object is examined, verified, and transition decisions made. Within each state, the guidelines provide for an initial walkthrough, intermediate walkthroughs, and a final walkthrough. Depending upon the progress of the development item, there may be one or many walkthroughs held within a state until the Approval to Proceed directs a state transition.

Each walkthrough has a well defined list of entry and exit requirements. In most cases, the exit requirements consist of Walkthrough Minutes, an Approval to Proceed, and the actual Software Development Item products.

Metrics are collected throughout the process to capture the return on investment in terms of the resources required to conduct the walk- throughs and the number of defects uncovered. At the beginning of each walkthrough, every reviewer is polled for the amount of time they spent in review.

Walkthroughs are the heart of the process. It is important to maximize the effectiveness of the walkthrough and minimize the overhead associated with it. The most important part of the walkthrough is the inspection that is conducted upon the development item. This means that the success of the walkthrough depends mainly upon the professional and conscientious examination of the walkthrough inputs by the reviewers. The objective is to uncover every defect. Reviewers are assigned at the start of the process and they remain the same throughout.

Walkthrough Metrics: Gathering metrics is an important part of the process and that includes metrics associated with the walkthrough effort. These metrics include: walkthrough duration; number of participants; external review time; pre and post-walkthrough overhead; major and minor defects/issues uncovered. Major defects are defined as those defects that would have negatively affected the operational system. Minor defects are anything else.

Quality Assurance Activities: One of the primary objectives of the State-Based Development Process is to engage every member of the team in quality assurance activities and make it a natural part of their development paradigm. We were convinced that the only way to insure quality in the end product was to make sure the process builds it in from the beginning to the end. If the process is followed, it will naturally lead to a higher quality product. However, project management must continually monitor and evaluate this aspect of the process to insure that good quality assurance efforts are being conducted.

Results: The State-Based Development Process was used throughout the ATAMS project and with great success, both from a productivity and a quality standpoint. Some sample metrics that characterize the walkthroughs are shown in Figure 2. One can discern that the return on our investment is 2.5 major defects found and over 6 minor defects uncovered. That equates to a little over one hour of staff time per defect when considering all cost and both major and minor defects.

From a quality/productivity standpoint, our results have been fairly consistent from the initial SCAI project through the ATAMS phases to-date. The statistics in Figure 3 are from the first operational release of ATAMS. Of the four Incident Reports that have been written for ATAMS, only one is applicable to the over 37, 000 hand-developed lines of code.

Management visibility was greatly improved because the development process was driven down in its granularity and status could be more readily seen.

To the project manager, the process has offered a number of benefits:

• Easily understood by managers and engineers; easily taught and caught, one-page graphic representation
• Provides clear communication of status within the team and to management
• Provides concise/definable states with clear entry/exit criteria, activity steps, procedures and methodologies
• Natural points and clear guidance for valuable capture of metrics
• Promotes łteam˛ concept and ownership
• Facilitates continuous process improvement
• Greatly increased the quality of the product