The objective of the Cleanroom methodology is to achieve or approach zero defects with certified reliability. As described by Hausler (1994), the Cleanroom methodology provides a complete discipline within which software personnel can plan, specify, design, verify, code, test and certify software. In a Cleanroom development, correctness verification replaces unit testing and debugging. After coding is complete, the software immediately enters system test with no debugging. All test errors are accounted for from the first execution of the program with no private testing allowed. As opposed to many development processes, the role of system testing is not to test in quality; the role of system testing is to certify the quality of the software with respect to the systems specification. This process is built upon an incremental development approach. Increment N+1 elaborates on the top down design of increment N . The Cleanroom process is built upon function theory where programs are treated as rules for mathematical functions subject to stepwise refinement and verification. Cleanroom specifications and designs are built upon box structure specifications and design. Box structure specifications begin with a black-box specification in which the expected behavior of the system is specified in terms of the system stimuli, responses and transition rules. Black boxes are then translated into state-boxes which define encapsulated state data required to satisfy black box behavior. Clear box designs are finally developed which define the procedural design of services on state data to satisfy black box behavior. Team reviews are performed to verify the correctness of every condition in the specification. During the specification stage an expected usage profile is also developed, which assigns probabilities or frequency of expected use of the system components. During system correctness testing, the system is randomly tested based on the expected usage of the system. In this process, software typically enters system test with near zero defects. The Cleanroom process places greater emphasis on design and verification rather than testing. In this process errors are detected early in the life cycle, closer to the point of insertion of the error.
-
EDUCATION, TRAINING, COURSES, AND CONFERENCES
Where to go for Cleanroom Software Engineering information.
-
Cleanroom Course from DACS.
< http://www.dacs.dtic.mil/training/cleanroom.shtml > This 1-day
course on Cleanroom Software Engineering, sponsored by the DACS and
taught by Rick Linger, can be held at your site.
- Cleanroom Courses
from SET. < http://www.toolSET.com/set_courses.html > Individual
courses from Software Engineering Technology in Cleanroom Process,
Specification, Development, Certification and an Overview.
-
Cleanroom Tutorial.
< http://www.stormingmedia.us/55/5569/A556982.htmlloral/cleanroom/tutorial/cleanroom.html >
A DoD STARS Tutorial by Software Engineering Technology, Inc.
-
LITERATURE
Literature and Bibliographies for Cleanroom Software Engineering.
-
A Business Case for Software Process Improvement: A DACS
State-of-the-Art Report
< http://www.dacs.dtic.mil/techs/roi.soar/soar.html > The purpose of
this State of the Art Report (SOAR) is to provide the details necessary
to rationalize, from a business perspective, investing in and
performing software process improvement. ROI Computation is discussed.
Cleanroom Software Engineering, Inspections, Reuse and Full SPI are
discussed in this report. Success Stories from Government use of SPI
are discussed.
-
A Cleanroom Bibliography
< http://iese.iese.fhg.de/home/bunse/reference.html > This
bibliography is maintained by Christopher Bunse, a researcher at
Fraunhofer-Einrichtung fr Experimentelles Software Engineering (IESE).
-
Bibliography.
< http://www.stormingmedia.us/55/5569/A556982.htmlloral/cleanroom/tutorial/reading.html >
Cleanroom Bibliography from the STARS Program.
-
CLEANROOM SOFTWARE ENGINEERING: The Picatinny Experience, by S. Wayne
Sherer. < http://lcsehp2.pica.army.mil/cleanroom/cleanroom.html >
This paper presents the experience, return on investment (ROI) and
lessons learned from a Cleanroom Software Engineering (CSE) based
process improvement at the Tank-automotive and Armaments Command
(TACOM) Life Cycle Software Engineering Center (LCSEC) at Picatinny
Arsenal.
- Cleanroom
Bibliography. < http://www.sni.net/~deckm/crbib.html > The
following is a selected bibliography of articles and reports on
Cleanroom software engineering from Michael Deck of Cleanroom Software
Engineering.
-
Experiences from Cleanroom Software Engineering
< http://www.cs.utk.edu/~trammell/CRT/experience.html > The following
are a few of the Cleanroom projects that have been publicly reported in
the literature, at conferences, or in WWW sites.
-
STARS Cleanroom Documents (Downloadable)
< http://source.asset.com/WSRD/indices/domains/CLEANROOM_SOFTWARE_ENGINEERING.html >
STARS project documents detailing the implementation of demonstration
projects using Cleanroom Software Engineering processes.
-
STARS Guide to Integration of Object Oriented Methods and Cleanroom
Software Engineering (William Ett & Carmen Trammell).
< http://www.stormingmedia.us/55/5569/A556982.htmlloral/cleanroom/oo/oo.html > The
STARS Object-Orientation Cleanroom Integration Study was performed to
provide (1) a resource for systems and software engineers to identify
how current object-oriented methods might be augmented by the practices
and methods of Cleanroom software engineering, and (2) an example of
integrating the techniques of an object-oriented method with those of
Cleanroom software engineering. The results of this study are described
in three products (1) The STARS Report: Guide to Integrating
Object-Oriented Methods with Cleanroom Software Engineering (Revised),
(2) The STARS Report: Integrating Objectory Use Case Modeling with
Cleanroom Specification Techniques, and (3) The Objectory/Cleanroom
Specification Practices Integration Task Work Products. The results of
object-oriented software development projects in the DoD have been
disappointing relative to the promise that OO technology seems to hold.
This report explores the feasibility of combining object-oriented
methods (well known for their focus on reusability) with Cleanroom
software engineering (well known for its emphasis on reliability) to
define a software process capable of producing results that are not
only reusable, but predictable and of high quality.
-
SERVICE PROVIDERS AND CONSULTANTS. A number of service providers and consultants are shown here.
- Cleanroom Software Engineering,
Inc. < http://www.sni.net/~deckm > Cleanroom Software
Engineering, Inc. was formed in 1993 to provide services to projects
seeking to use all or part of the Cleanroom approach. It specializes in
adapting Cleanroom to solve unique software-process problems. Recent
engagements have applied Cleanroom to object-oriented development,
legacy systems enhancement, and hard real-time aerospace applications.
The company offers process and work-product assessment, training,
consultation, and software-development services. It is committed to
using the Cleanroom principles as the basis for a flexible, tailored
software development process.
- Q-Labs.
< http://www.q-labs.com/> Q-Labs is a leading international
professional services company in the Software Engineering sector. We
are change facilitators providing state of the art solutions to
industry. Examples of core technologies in Q-Labs current portfolio
include Cleanroom Software Engineering, Reuse, Capability Maturity
Model, Experience Factory, Quality Improvement Paradigm and Cleanbase.
We provide our clients with top-quality leading edge solutions that are
designed to keep them steps ahead.
- Software Engineering Technology,
Inc. < http://www.toolSET.com/> SET is a technology firm
specializing in Cleanroom software engineering. SET was founded in 1987
by Harlan Mills, the inventor of the Cleanroom concept. The SET
Cleanroom engineering approach is based on a quarter century of
research, development, field testing and refinement. It is the catalyst
for bringing scientific principles, engineering practices and
management control to the software industry.
- University of
Tennessee Software Quality Research Laboratory.
< http://www.cs.utk.edu/~trammell/SQRL/> The University of Tennessee
Software Quality Research Laboratory does applied research and
development in software engineering. Industrial sponsors support
software quality research, demonstration, and technology transfer
activities.
-
SOFTWARE TOOLS Tool providers for the Cleanroom Software Engineering Community
-
Statistical Test Case Generation Facility.
< http://www.clearlake.ibm.com/MFG/solutions/cleanrm.html#stgf> This
page is under construction. Cleanroom testing is accomplished for
successive code increments by executing test cases randomized against
the projected user input distribution. Statistical sampling is used to
create the test cases based on the usage specification, and statistical
methods are used to perform the reliability analysis using a
reliability model. In addition to the customer usage distribution,
other alternative usage distributions may be specified as required for
subsequent independent certification. For example, an alternative usage
distribution might be specified to stress infrequently invoked
functions whose failure would result in severe consequences
- CleanTest: A Tool For
Cleanroom Testing < http://www.sni.net/~deckm/cleantes.html >
CleanTest is a brand-new tool to support the statistical testing
process used in Cleanroom software engineering. It was developed by
Cleanroom Software Engineering, Inc., a company whose entire business
is helping organizations use Cleanroom techniques. CleanTest solves the
problem of recording and maintaining operational profiles (usage
specifications) with an easy-to-use, graphical user interface. Its
intuitive model simplifies the creation of profile data and the
evolution of that data throughout th development process. With a few
keystrokes, CleanTest can generate hundreds or thousands of statistical
test cases based on your input profile. CleanTest can keep track of
what parts of the operational profile have been tested, a valuable aid
to effective testing.
-
Cleanroom Certification Model.
< http://www.clearlake.ibm.com/MFG/solutions/cleanrm.html#ccm > A
number of statistical models have been developed to predict the
ultimate reliability of a software product during software testing.
These models use the expected improvement with usage testing (called
reliability growth) to project the reliability at a later point in
testing.
-
Cleanroom Software Engineering < http://wwwagse.informatik.uni-kl.de/STTI/CR/TECHNO/cleanroom.html > A page under construction by the Software Engineering Group at the University of Kaiserslautern.
- SW-Cleanroom E-Mail Discussion List < http://www.quality.org/lists/sw-cleanroom.info.txt> Information about an E-mail discussion list for Cleanroom Software Engineering, including how to subscribe.
-
CLEANROOM SOFTWARE ENGINEERING: The Picatinny Experience, by S. Wayne Sherer. < http://lcsehp2.pica.army.mil/cleanroom/cleanroom.html > This paper presents the experience, return on investment (ROI) and lessons learned from a Cleanroom Software Engineering (CSE) based process improvement at the Tank-automotive and Armaments Command (TACOM) Life Cycle Software Engineering Center (LCSEC) at Picatinny Arsenal.
- Cleanroom Bibliography. < http://www.sni.net/~deckm/crbib.html > The following is a selected bibliography of articles and reports on Cleanroom software engineering from Michael Deck of Cleanroom Software Engineering.
- Experiences from Cleanroom Software Engineering < http://www.cs.utk.edu/~trammell/CRT/experience.html > The following are a few of the Cleanroom projects that have been publicly reported in the literature, at conferences, or in WWW sites.
-
Objectory/Cleanroom Specification Practices Integration Task Work Products. < http://www.asset.com/stars/loral/cleanroom/oo/specintg.htm > The work products include the requirements set for the study and complete black box and use case specifications that address and elaborate those requirements.
-
STARS Cleanroom Documents (Downloadable) < http://source.asset.com/WSRD/indices/domains/CLEANROOM_SOFTWARE_ENGINEERING.html > STARS project documents detailing the implementation of demonstration projects using Cleanroom Software Engineering processes.
- STARS Guide to Integration of Object Oriented Methods and Cleanroom Software Engineering (William Ett & Carmen Trammell). < http://www.stormingmedia.us/55/5569/A556982.htmlloral/cleanroom/oo/oo.html > The STARS Object-Orientation Cleanroom Integration Study was performed to provide (1) a resource for systems and software engineers to identify how current object-oriented methods might be augmented by the practices and methods of Cleanroom software engineering, and (2) an example of integrating the techniques of an object-oriented method with those of Cleanroom software engineering. The results of this study are described in three products (1) The STARS Report: Guide to Integrating Object-Oriented Methods with Cleanroom Software Engineering (Revised), (2) The STARS Report: Integrating Objectory Use Case Modeling with Cleanroom Specification Techniques, and (3) The Objectory/Cleanroom Specification Practices Integration Task Work Products. The results of object-oriented software development projects in the DoD have been disappointing relative to the promise that OO technology seems to hold. This report explores the feasibility of combining object-oriented methods (well known for their focus on reusability) with Cleanroom software engineering (well known for its emphasis on reliability) to define a software process capable of producing results that are not only reusable, but predictable and of high quality.
- STARS Report: Integrating Objectory Use Case Modeling with Cleanroom Specification Techniques < http://www.asset.com/stars/loral/cleanroom/oo/sect1.htm > This report describes applying use case modeling and black box analysis to a common set of requirements, and presents an integrated Objectory-Cleanroom method for preparing a behavioral specification for a software system.
