The McGarry effort is sponsored by the Joint Logistics Commander Computer Resources Management Committee, in short the JLC-CRM. The JLC-CRM also sponsors the 498 revision to 2167A. McGarry has formed a committee of DoD and industry personnel which has been meeting for several months.

The objectives of the committee's effort are to:

• Identify, based upon practical experience, the software measurement data and associated metrics which best support the implementation of MIL-STD-498
• Prioritize the identified software data and metrics for implementation under MIL-HDBK-SWM
• Define the requirements for implementing a viable software measurement process within the program organization.

The Handbook is outlined in four parts:

• Development Program Objectives
• Measurement Implementation Methodology
• Measurement Data and Metrics Requirements
• Contract Requirements

The basic points, or guidelines of the Handbook are that data and metrics will be required by all programs and the Handbook will be targeted at inexperienced metrics users. It will include basic management and product metrics. The metrics being targeted are those that are in widespread use and easily understood and analyzed. There are also enhanced guidelines for more experienced users who have a requirement for a more sophisticated metrics program. The effort includes a plan for an advanced version of the guidebook which would be published at a future date.

There are a variety of measurement rules that are under consideration. Some of these are: analysis based on low level software attribute data, all data to be derived or collected on a periodic basis and in a consistent manner, a flexible metrics implementation based on program objectives and data availability, and a focus on single program analysis, not cross program comparison.

In the current thinking the metrics would be structured in five measurement categories to include: Milestone Performance, Schedule Performance, and Work Unit Completion in the Progress area; Effort Allocation, Staff Allocation, Cost Performance and Facility Utilization in the Resources area; Defect Profiles and Product Stability (Requirements and Engineering Change Proposals (ECPs)) in the Product Quality area; Size (pages of documentation, Software Design Units, etc.), Computer Resources, Complexity, and Reliability in the Product area; and Capability, Performance and Environment in the Process area.

The committee is considering a wide range of issues to include:

• At what level in the project is data collected
• Defining the scope of program application
• Tailoring data and metrics requirements to specific issues and characteristics
• How to place measurement requirements on contract
• Implementing an overall quantitative analysis approach
• How to collect and validate software data
• How to process and manage software data
• How to choose and apply useful metrics
• How to interpret the data and associated metrics
• Key metrics relationships
• Evaluating program objectives
• Metrics limitations

While there is some concern that MIL-STD-498 may not be published (due to the recent OSD policy announcement on increased use of commercial practices (see page 3 of this newsletter "Two New Memoranda Issued by the DoD")) the Committee believes that the guidebook will stand on its own for the implementation of a measurement program. The actual construction of the guidebook, once the committee finishes its work, will be under contract to Logicon, Inc.

This article was prepared from development materials of the Metrics HDBK working committee and does not represent the position of the committee or the Joint Logistics Commanders Computer Resources Management Coordinating Group. For information on this program contact:

John J. McGarry
Naval Undersea Warfare Center, Division Newport
Code 2233, Bldg. 1171/2
Newport, RI 02841
(401) 841-3834, Fax: (401) 841-2130
[email protected]