Software Tech E-News: Measurement

SOFTWARE MEASUREMENT

Techniques or methods that apply software measures to software engineering objects (e.g. code, inspections) to achieve predefined goals. A measure is a mapping from a set of software engineering objects to a set of mathematical objects. Measurement goals vary with the software engineering object being measured, the purpose of measurement, who is interested in these measurements, which properties are being measured, and the environment in which measurement is being performed. Examples of measures include software size, Halstead's software science measures, and McCabe's cyclomatic complexity. Associated models include sizing models, cost models, and software reliability models.

Table of Contents:
Complexity Metrics DACS Measurement Services Education and Training Literature Articles and Papers Bibliographies Technical Reports	People (experts) Programs/Organizations Tools Related Topics: Cost Estimation Function Points

COMPLEXITY METRICS - Cyclomatic complexity, Halstead metrics, Design measures, etc.

McCabe & Associates (http://www.mccabe.com/) McCabe & Associates was founded by Tom McCabe, Sr., the president and inventor of a widely used complexity metric. McCabe & Associates is the developer of the McCabe Visual ToolSets (TM). This includes The Visual Quality ToolSet, The Visual Testing ToolSet, and The Visual Reengineering ToolSet.
Metre v2.3 (http://www.lysator.liu.se/c/metre-v2-3.php) Metre is a freely distributable ANSI/ISO Standard C parser. Reports Halstead metrics, various line and statement counts, backfired function points, control depth, identifier count, number of functions and modules, and a call graph.
Programs for C Source Code Metrics (http://www.cs.queensu.ca/Software-Engineering/archive/static.php) Some free programs to count lines of code, cyclomatic complexity, Halstead metrics, backfired Function Points, etc. for C code. The tools can be compiled on SunOS.
SET Laboratories, Incorporated (http://www.molalla.net/~setlabs/) SET Laboratories, Inc. has been a leading provider of software measurement products and services since 1987.
Telelogic (http://www.telelogic.com/) A developer and vendor of CASE tools, including the measurement tool Logiscope and object oriented tools.

DACS MEASUREMENT SERVICES

DACS Measurement Services Brochure This brochure describes the DACS consulting services using the "Goal/Question/Metric (GQM)" method developed by Dr. Victor Basili, to precisely define the measurement process.

This is a link to download a brochure in PDF format.
DACS Product & Services The DACS technical area of focus is software technology and software engineering, in its broadest sense. DACS supports the development, testing, validation, and transitioning of software engineering technology.

EDUCATION, TRAINING, COURSES, AND CONFERENCES - Software Measurement Training resources

Carnegie Mellon -Distance Education: Software Measurement (http://www.distance.cmu.edu/SE/measure.php) This course will introduce students to applying software measurement ‹ from need identification through analysis and feedback ‹ into the process. Much of the course material used to demonstrate the concepts are based on how software measurement is used by managers and practitioners in industry today. The content of the course is taught within the framework of the software engineering process.
Conferences on Software Metrics, Measurement, and Quality (http://www.dciexpo.com/) This site provides a calendar of software metrics, measurement, and quality conferences. Important dates are included with each citation.
Digital Consulting, Inc. (http://www.dciexpo.com/) DCI hosts conferences and seminars, including one on Function Point Analysis.
PSM -- A Structured Measurement Course (https://www.thedacs.com/training/practical_swm.php) Taught by John J. Marciniak, this course will present an introduction to measurement as practiced in the DoD, with a special focus on 'Practical Software Measurement -- A Guide to Objective Program Insight' (PSM).
Quality Plus Technologies Guide to Software Measurement Start-Up (http://www.qualityplustech.com/t_guide_smea.php) This one-day workshop provides an introduction to software measurement and is intended for IS managers, measurement coordinators, and others who require an appreciation of the key elements to starting and implementing a software measurement program.
SEI Course - Implementing Goal-Driven Software Measurement (http://www.sei.cmu.edu/products/courses/implement.goal-driven.sw.meas.php) This course is for managers and practitioners who want to learn how to define and use goal-driven measures to manage and improve software projects, products, and processes. Participants receive a guidebook -- with clearly identified steps and templates that enables them to use and lead goal-driven software measurement in their own organizations.
SEI Course - Managing Software Projects with Metrics (http://www.sei.cmu.edu/products/courses/man-swdev-metrics.php)- Managing Software Projects with Metrics course is designed for Project Managers, or individuals supporting Project Managers of software-intensive systems development, maintenance, and acquisition. The objective is to teach project managers how software measurement can help them meet their goals.
STSC Metrics Workshops (http://www.stsc.hill.af.mil/pns/pnsindex.php#measure> A listing of Metrics Workshops offered by the Software Technology Support Center.
Software Measurement: Implementation and Practice (https://www.thedacs.com/training/soft.meas.php) Taught by Dr. Victor Basili, this course will highlight Measurement and Metrics, Measurement Paradigms, and the Experience Factory. This course is designed for the software professional involved in project management, oversight for software intensive projects, software acquisition management, or software development and engineering who has experience with software and software development but is not familiar with measurement or measurement practice.
Software Quality Institute (SQI) (http://www.lifelong.engr.utexas.edu/sqi/index.cfm) The SQI is a multidisciplinary partnership between The University of Texas at Austin and the software and information systems organizations in Texas. Its mission is to inform and educate software producers and software users at the local, state and national levels about issues vital to the production and application of high-quality software. Extensive training offered to software professionals includes 48-week Software Project Management Certificate Program. About 30 seminars a year are offered on a variety of SW Measurement topics.

SOFTWARE MEASUREMENT LITERATURE - Reports, articles, bibliographies and short papers relating to Software Measurement and Metrics.

Technical Reports

A History of Software Measurement at Rome Laboratory (https://www.thedacs.com/techs/history/title.php) A DACS report surveying past and contemporary software metrics research at Rome Laboratory.
Modern Empirical Cost and Schedule Estimation Tools (https://www.thedacs.com/techs/estimation/title.php) New software development philosophies and technologies have emerged in the 1980s and 1990s to reduce development costs and improve quality of software products. These new approaches frequently involve the use of Commercial-Off-The-Shelf (COTS) software, software reuse, application generators, and fourth generation languages. The purpose of this paper is to identify, discuss, compare and contrast software cost estimating models and tools that address these modern philosophies.
Present Value of Software Maintenance: A DACS State-of-the-Art Report Deciding to engage in a software project typically results in incurring costs and generating revenues over a long time period. Introducing new technology into the software process can likewise be considered an investment decision. This paper presents capital budgeting techniques employed among financial analysts and upper-level management to evaluate such investment decisions.
Quantitative Methods Report (http://www.cs.mtu.edu/qmtc/q-methods.php) The IEEE Computer Society Technical Council on Software Engineering (TCSE) Committee on Quantitative Methods was formed to foster communications between quantitative methods researchers and practitioners. This Web site provides access to PostScript versions of the committee's newsletters, conference announcements, and some tool information.
Software Error Analysis (http://hissa.ncsl.nist.gov/HHRFdata/Artifacts/ITLdoc/209/error.php) This report, from the National Institute of Standards and Technology, provides guidance on software error analysis. Error detection techniques considered in the study are those used in software development, software quality assurance, and software verification, validation and testing activities. These techniques are those frequently cited in technical literature and software engineering standards or those representing new approaches to support error detection.
Software Measurement Guidebook v2.0 (http://www.ntis.gov/fcpc/cpn5671.php) This Guidebook provides practical guidance for the quantitative management of a software development project. It describes how to use measurement methods in setting quantifiable goals for a software project, in selecting metrics to support those goals, and in managing to meet those goals. It presents methods for estimating software size, cost, and development schedule. It also presents methods to estimate the costs of support to software development. This guidebook describes the Software Engineering Institute (SEI) process maturity model and the role of software metrics in raising the capability maturity level of a software development organization.
Statistical Software Engineering (http://www.nap.edu/readingroom/books/statsoft/) A report of the Panel on Statistical Methods in Software Engineering, of the Committee on Applied and Theoretical Statistics, Board on Mathematical Sciences, of the National Research Council. The NRC operates under the charter granted to the National Academy of Sciences by the U. S. Congress.

Articles and Papers

A Comparative Study of Two Techniques for Analyzing Software Measurement Data (http://citeseer.nj.nec.com/did/61774) In this paper, two data analysis techniques - Rough Sets and Logistic Regression - are compared, from both the theoretical and the experimental points of view. In particular, the empirical study was performed as part of the ESPRIT/ESSI project CEMP on a real-life maintenance project, the DATATRIEVE project carried out at Digital Engineering Italy.
A Software Metrics Primer (http://www.processimpact.com/articles/metrics_primer.php) In this article, Karl E. Wiegers describes some basic software measurement principles and suggest some metrics that can help you understand and improve the way your organization operates. (originally published in Software Development, July 1999)
A Synopsis of Seven Measurement Initiatives- DACS Newsletter (Spring 1996) by John Marciniak, There are a number of initiatives and programs underway in the DoD. Some of them have been in place for some time while others are of recent vintage.
Agena - Bayesian Networks and Software Metrics Resources (http://www.agena.co.uk/downloads/whitepapers.php) This site provides access to a collection of articles that give an overview of software quality, software metrics, and Bayesian Belief Networks.
DACS Book Review: Practical Software Metrics for Project Management and Process Improvement by Robert B. Grady - DACS Newsletter (Winter 1993) This book (1992) is the successor to the 1987 book, Software Metrics: Establishing a Company-Wide Program, by R. Grady and D. Caswell. The earlier book described Hewlett-Packard's (HP) effort to establish a company-wide software metrics program. Grady's newer book reflects HP's more recent experiences, without requiring familiarity with the earlier book.
DoD Software Tech News (STN 1-2): Measurement The DoD Software Tech News provides cutting-edge, topical Software Technology information. This issue contains four articles on Software Measurement.
- Earned Value(EV) and Automated Information Systems(AIS) - by Fred Manzer, Defense Systems Management College
- Analyzing Quantitative Data Via the Web, Summary of the paper by Robert L. Vienneau, DACS and James D. DeLude, Kaman Sciences
- Practical Software Measurement (PSM) 1997 Users' Group Conference by Robert Vienneau, DACS and Jack McGarry, Naval Undersea Warfare Center (NUWC)
- Software Engineering Measurement and Analysis Initiative at the SEI by David Zubrow, SEI
Empirical Software Engineering: An International Journal (http://kapis.www.wkap.nl/kapis/CGI-BIN/WORLD/journalhome.php?1382-3256> This journal provides a forum for researchers and practitioners to report both original and replicated controlled experiments, field studies, and qualitative studies. The Web page lists the editors, provides access to ordering information, etc.
From Software Metrics to Software Measurement Methods: A Process Model (http://www.lrgl.uqam.ca/publications/pdf/208.pdf)This paper by Jean-Philippe Jacquet and Alain Abran of UQUAM, presents a process model for software measurement methods. The proposed model details the distinct steps from the design of a measurement method, to its application, then to the analysis of its measurement results and last to the exploitation of these results in subsequent models, such as in quality and estimation models.
Insight -- the Army's Software Metrics Newsletter (http://www.armysoftwaremetrics.org/products/insight.asp)This newsletter, published by the Army Software Metrics Office (AMSO), provides news and information on the Army software metrics initiatives. Back issues are available in PDF and Word formats.
McCabe and Associates -- Software Metrics This page lists McCabe's metrics and provides a brief description of each. A reference to "Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric" by Arthur Watson and Tom McCabe is provided.
Measuring and Evaluating Maintenance Process using Reliability, Risk, and Test Metrics (https://www.thedacs.com/topics/reliability/IEEETrans.pdf) Written by Norman F. Schneidewind, Naval Postgraduate School, this article first appeared in IEEE Transactions on Software Engineering, Volume 25, Number 6, November/December 1999, pp. 768-781. [This link leads to a PDF file]
Metrics for Object-Oriented Software Engineering Written by Edward V. Berard, The Object Agency, this article describes the difference between Object-Oriented and other Software Engineering Metrics.
Return on Investment from a Software Measurement Program (http://members.aol.com/geshome/ibelieve/ROI/ROI.php) The Missile Warning and Space Surveillance Sensors (MWSSS) program management office became responsible for the software maintenance of seven products in November 1994. This paper describes the software release planning and implementation process and the measurement strategy. It examines the costs of developing and sustaining the measurement program over two years as well as quantitative and qualitative returns on that investment.
Software Metrics Collection Techniques for Product Assessment (http://www.comp.nus.edu.sg/~wanghao/frameVersion.php) This article lists various metric-based methods of assessing software quality based on quality attributes.
Software Metrics Sites: Papers (http://user.cs.tu-berlin.de/~fetcke/ metrics/papers.php) The Software Metrics Sites guide to Internet resources on software measurement, process improvement and related areas.
Software Metrics: Ten Traps to Avoid (http://www.processimpact.com/articles/mtraps.php) While software metrics can help you understand and improve your work, implementing a metrics program is a challenge. This paper by Karl E. Wiegers, Process Impact discusses the technical and the human aspects of software measurement management.

Bibliographies

NASA Software Engineering Laboratory (SEL) (http://sel.gsfc.nasa.gov/website/documents/sec06.php) Annotated Bibliography of SEL Literature This collection briefly describes NASA SEL measurement reports.
Object-Oriented Metrics: People and Publications (http://www.sbu.ac.uk/~csse/publications/OOMetrics.php) This annotated bibliography is meant to offer an authoritative coverage of what is known about measurement of object-oriented technology and its use. It was prepared by Robin Whitty of the Centre for Systems and Software Engineering at South Bank University, London.
Rome Laboratory Research in Software Measurement - A DACS Custom Bibliographic Search (http://www.rome.kaman.com/techs/bib/bib.php) A DACS Annotated bibliography. This bibliography was prepared as input to the DACS project that produced A History of Software Measurement at Rome Laboratory.
Software Metrics - A subdivided bibliography (http://irb.cs.uni-magdeburg.de/sw-eng/us/bibliography/bib_main.php) A bibliography produced by Dr. Reiner Dumke of the Otto von Guericke University of Magdeburg. It seems fairly comprehensive, although the author warns that it reflects the literature available to him. This bibliography would be improved by including annotations and more introductory material.
Software Metrics Sites: Bibliographies (http://user.cs.tu-berlin.de/~fetcke/metrics/bib.php >The Software Metrics Sites are a guide to Internet resources on software measurement. This section is a pointer to -online bibliographies.
The University of Melbourne -- A Literature Survey of Tailoring in Software Measurement (http://www.cs.mu.oz.au/tr_submit/test/cover_db/mu_TR_2000_5.php) Although tailoring of software methodologies has become a major feature in recent years, how this effects measurement is not always clear. Different sources have a different perspective on tailoring of measurement in software methodologies.

SOFTWARE MEASUREMENT PEOPLE - A listing of people prominent in software measurement who have Web pages.

Basili, Victor (http://www.cs.umd.edu/users/basili/) A professor at the University of Maryland, Dr. Basili is a noted advocate and practitioner of quantitative methods in software engineering. He was a co-founder of NASA's Software Engineering Laboratory and has won numerous awards. His Web page provides pointers to the journals and organizations with which he is associated, as well as papers and other descriptions of his research interests.
Fenton, Norman (http://www.csr.city.ac.uk:8080/people/norman.fenton/) Professor Fenton has been at the Centre for Software Reliability, City University, London since 1989. He recommends a formal measurement-theory approach to software measurement.
Harrison, Warren (http://www.cs.pdx.edu/~warren/) Dr. Harrison is a professor at Portland State University, and a past Department Chair. His primary interest is software engineering, particularly software testing and measurement. This page seems designed for Dr. Harrison's own use.
Kemerer, Chris (http://www.pitt.edu/~business/faculty/kemerer.php) The David Roderick Chair in Information Systems at the University of Pittsburgh, Dr. Kemerer's research interests include management and measurement issues in information systems and software engineering. His Web page overviews his work, provides access to some of his papers and lists others, and includes other links.
Littlewood, Bev (http://www.csr.city.ac.uk/people/bev.littlewood/) Professor Littlewood is known for critical work in software reliability, including the development of models and modelling techniques to illustrate his theories. He founded the Centre for Software Reliability, of which he is Director, and is Professor of Software Engineering at City University (London). This Web page is basically a terse resume.
McCabe & Associates (http://www.mccabe.com/) McCabe & Associates was founded by Tom McCabe, Sr., the president and inventor of a widely used complexity metric. McCabe & Associates is the developer of the McCabe Visual ToolSets (TM). This includes The Visual Quality ToolSet, The Visual Testing ToolSet, and The Visual Reengineering ToolSet.
Selby, Richard (http://sunset.usc.edu/COCOMOII/Researchers/selby.php) Dr. Selby is a associate professor at the University of California, Irvine. He has written numerous papers on empirical analysis of software and is on the editorial board of several journals. He founded Amadeus Software Research, Inc. to develop and market software CASE tools, and he is a member of serval software measurement organizations. Currently, this Web page is quite sparse.
Zelkowitz, Marvin (http://www.cs.umd.edu/users/mvz/index.php) Dr. Zelkowitz is a professor at the University of Maryland. His research interests include experimentation in software engineering, environments, formal methods, technology transfer, and program complexity. His page gives an overview of his research and pointers to interesting sites.
Zuse, Horst (http://www.cs.tu-berlin.de/~zuse/index.php) Parts of this page are in German. Dr. Zuse is an advocate of applying measurement theory to software metrics. His page provides some of his papers, information about his books, and instructions on accessing a database on software complexity. This page is more a resume in hypertext, although good pointers are provided to other sites.
Zweben, Stuart (http://www.cis.ohio-state.edu/~zweben/) Dr. Zweben is past President of ACM and Chair of the Department of Computer and Information Science at Ohio State University. His interests include the testing of object-based software and empirical assessment of the effectiveness of various software engineering principles and practices. His Web page includes links to various organizations with which he is affiliated.

PROGRAMS AND ORGANIZATIONS Software measurement programs and organizations

Boston Software Process Improvement Network (SPIN) (http://www.cs.uml.edu/Boston-SPIN/) Provides information on the Boston SPIN and information useful for Software Process Improvement (SPI), including back issues of the Boston SPIN newsletter and a list of Internet resources useful for SPI.
Practical Software Measurement (PSM) (http://www.psmsc.com/) Practical Software Measurement (PSM) is sponsored by the Joint Logistics Commanders Joint Group on Systems Engineering. The goal of the project is to provide Program Managers with the objective information needed to successfully meet cost, schedule, and technical objectives on software intensive programs.
Software Engineering Institute (SEI) Software Engineering Measurement and Analysis (SEMA) Program (http://www.sei.cmu.edu/sema/welcome.php) Provides information on the SEI software measurement program, including their definitions of size, quality, and cost and their recommended core measures.
Software Engineering Institute (SEI) Software Process Improvement Network (SPIN) (http://www.sei.cmu.edu/collaborating/spins/spins.php) The Software Process Improvement Network is comprised of individuals who want to improve software engineering practice. The individuals are organized into regional groups called 'SPINs' that meet and share their experiences initiating and sustaining software process improvement programs. This page, provided by the Software Engineering Institute, contains directories of U.S. and international SPINs, as well as information about how to start a SPIN in coordination with the SEI.
TeraQuest Metrics, Inc. (http://www.teraquest.com/) TeraQuest is a full-service provider of process improvement capability, including process assessment using the SEI's Capability Maturity Model (CMM) for Software, People Capability Maturity Model (P-CMM), process assessments using the Systems Engineering Capability Maturity Model (SE-CMM), risk assessment, requirements elicitation, and training.
United States Army Software Metrics System (http://www.armysoftwaremetrics.org/) The U.S. Army has 14 software metrics that assist program managers in developing software on time and within the allotted budget. This web site features these 14 metrics...and much more. It contains the latest products that the Army Software Metrics System (ASMS) Office releases, information on future training courses, the ASMS' quarterly newsletters, and information on the Army's metric policy updates.
Washington, DC, Software Process Improvement (SPIN) (http://www.software.org/DCSpin/) The DC SPIN is a leadership forum for the free and open exchange of software process improvement experiences and practical ideas. We promote achieving higher levels of process maturity, software quality, and mutual respect. Our goal is to help sustain commitment and enhance skills through an active program of networking, publications, recognition of excellence, and mutual support.

SOFTWARE MEASUREMENT TOOLS Software cost estimating tools.

Goel-Okumoto Software Reliability Model (https://www.thedacs.com/about/services/goel.php) An automated version of the Goel-Okumoto Nonhomogeneous Poisson Process Software Reliability Model which runs on an IBM-PC. Distributed by the DACS.
PRICE Systems (http://www.pricesystems.com) PRICE-S is a well-known software cost estimating model.
Quantitative Software Management (http://www.qsm.com/) Lawrence Putnam is the president of QSM. They distribute SLIM and related software cost modeling tools.
Software Productivity Research Information Center (http://www.spr.com/) A leading provider of software measurement, assessment, and estimation products and services. Capers Jones is SPR's chairman and founder.
COSMIC - NASA's Software Technology Transfer Center (http://www.openchannelfoundation.org/cosmic/) Open Channel Software has entered into an agreement with the National Technology Transfer Center (NTTC) to publish the COSMIC software collection. This collection represents software created by NASA in a wide range of disciplines including engineering, chemistry, aerodynamics, and other areas. The COSMIC collection consists of over 500 programs.
Process Focus Software (http://www.processfocus.com/) Process Focus Software provides software tools supporting process improvement activities, including an interactive interface to the Software Engineering Institute's Capability Maturity Model (CMM).
Quantitative Software Management (http://www.qsm.com/) Lawrence Putnam is the president of QSM. They distribute SLIM and related software cost modeling tools.
SET Laboratories, Inc. (http://www.molalla.net/~setlabs/) SET Laboratories, Inc. has been a leading provider of software measurement products and services since 1987.
Web-Integrated Software metrics Environment (WISE) (http://research.ivv.nasa.gov/projects/WISE/index.php) WISE is a WWW-based project management and metrics system available on the www. Programmers and managers can log issue reports, track the status of issues, and view project metrics using standard WWW browsers.