Sizing and Estimating for Real-time Software - the COSMIC-FFP Method

By Charles Symons, Common Software Measurement International Consortium

If a reliable method existed that allowed managers of real-time software development or enhancement projects to ….

A. estimate a software size from its requirements, early in the life of a project B. enter that size together with other parameters into formulae to estimate project effort and duration (where the formulae had been calibrated on recently-completed projects in the same domain) C. track the software size as requirements evolve (e.g. for contract control purposes) and compute project performance parameters (e.g. productivity, speed of delivery and defect density) when the project is delivered

…. then you would expect the method to be widely used.

The world of business application software has had such a process for more than 25 years, since the 1970s when Allen Albrect created ‘Function Point Analysis’ (FPA) [1].

This article describes a method called COSMIC-FFP (Common Software Measurement International Consortium – Full Function Points) that can be used for sizing ‘real-time’ software requirements, as well as those of business software, to provide the key ingredient for project estimating and project performance measurement. Under ‘real-time’ software we include process control, embedded, telecoms, infrastructure (i.e. operating system) software and such-like.

1. The Background to COSMIC and Its Aims Figure 1 of the Tech Views article (see page 2 of this issue) presents the history and evolution of Functional Size Measurement (FSM). It illustrates how a number of variants of Albrect’s FPA emerged and evolved over time and that, now, most of them have either ceased, or converged into two major methodologies, International Function Point Users Group (IFPUG)-sponsored FPA and COSMIC-sponsored FFP. IFPUG FPA and its variants are collectively known as ‘1st Generation’ (1G) FSM methods because they were designed for and embraced the business software paradigms and terminology of the 80s and 90s. As new software paradigms and languages evolved and even as the business world relies more heavily on real-time software, the scope of applicability of these 1G FSM methods is, perhaps, more limited. They remain strong and useful in the pure business application world, but are losing relevance for hybrid software environments as new functional sizing methods, such as COSMIC-FFP, emerge.

Lacking a functional sizing method that seemed appropriate, many large real-time environments have relied on estimating Source Lines of Code (SLOC) as the size measure and have built up large databases containing SLOC size measures for their projects. They perform the effort estimating step, either by formulae or by searching the database for analogues of the new project. However, although SLOC can be counted accurately and automatically upon project completion, estimating size in units of SLOC at the requirements stage is, obviously, not ideal.

With these aforementioned issues in mind, a group of software metrics experts gathered in late 1998 to form ‘COSMIC’, the Common Software Measurement International Consortium. Most of the founding members (including this author) were members of an ISO Working Group (ISO/IEC JTC1 SC7 WG12) on Functional Size Measurement that had laboured for several years to try to agree on some basic principles for FSM [3], but progress was slow. It became clear to some WG12 members that if ever we were to seriously improve on 1G FSM methods, this would have to be undertaken as a private initiative, outside the formal ISO processes.

An example of the sort of difficulty in this subject that WG12 faced is that there is still no definition of ‘functional’.

The ISO/IEC 14143 standard, published in 1998 [3], defines “Functional size” as:

“A size of the software derived by quantifying the Functional User Requirements”, where the latter are defined as “a sub-set of the users’ requirements …. (they) represent the user practices and procedures that the software must perform to fulfil the users’ needs. They exclude Quality Requirements and any Technical Requirements”.

This definition, with hindsight, has a strong flavour of the business application software world. For our purposes, think of “functional size” as a size measure concerned purely with requirements for information processing, thus excluding any requirements concerned with quality, technical or implementation factors. The latter must be taken into account when estimating project effort, but do not contribute to functional size.

When COSMIC was established in late 1998, it set itself a clear initial aim, namely ‘to develop, test, bring to market and seek acceptance of new software sizing methods to support estimating and performance measurement’. The methods should:

• Measure a ‘functional size’ of software
• Be applicable to business application and real-time software and hybrids of these
• Be applicable for software in any layer of a multi-layer architecture
• Be applicable at any time in a software project life-cycle from early requirements gathering through to post-implementation, and for maintenance and enhancement activities

Today, COSMIC is still a voluntary, ‘open’ organization, but now with representatives from 15 countries [4] on its International Advisory Council. The functional sizing method that has been developed is called COSMIC-FFP. Its stability is demonstrated by the fact that an International Standard for COSMIC-FFP [5] was published in 2003. The method is rapidly gaining acceptance in the market place (see side panel for examples of two current users of the method in the real-time domain).

2. Overview of the COSMIC-FFP method

Space does not allow a full description of the method in this article, so here we will give only a summary of the key concepts with some elaboration of particularly important concepts for those working in the real-time software domain. For the full definition of the method, including definitions of all the main concepts, see the ‘Measurement Manual’. This is freely available for download from the COSMIC web site in several languages [6].

The COSMIC-FFP Measurement Process consists of three main phases:
1) Setting the Measurement Strategy
2) Mapping the ‘Functional User Requirements’ (or ‘FUR’) of the software to be measured to the COSMIC-FFP concepts
3) Measuring the resulting COSMIC-FFP model of the FUR

Phase 1: Setting the Measurement Strategy

This phase consists of first establishing the Purpose and Scope of the measurement. The Purpose is important because it affects the accuracy that will be required of the measurement, the timing of the measurement in the software life-cycle, the software artefacts (statements of requirements, physical screens, etc) that must be used, and so on. The measurement Scope determines what functionality is included in any one measurement. It might be, for example, that if the Purpose is to support estimating for a distributed software system and if the various components are planned to be implemented on different types of technical platforms, then the FUR must be separated into a number of separate measurement Scopes, one for each platform.

The Purpose and Scope together determine the “Measurement Viewpoint” that will be adopted. This defines the form of abstraction used for the FUR and which will be used for the measurement1. In this short article, it is best described by examples.

• In the domain of simple real-time (for example, embedded) software, the FUR are normally described in terms of the functionality that must be provided to the ‘direct users’ of the software. These ‘direct users’ are typically hardware devices, e.g. sensors, buttons, display panels, valves etc., and also other pieces of software that interact directly with the software to be measured. They are called ‘direct’ users because there is no software between those users and the software to be measured. We call this abstraction the ‘Direct User’ measurement viewpoint2.
• In the domain of business application software, the FUR are normally described in terms of the functionality that must be provided to the human users of the software. All hardware (screens, keyboards, printers, etc) and software (operating system layers) between the human user and the application software to be measured is invisible in this abstraction and is ignored. We call this the ‘End User’ measurement viewpoint.

The COSMIC-FFP method can be applied to measure a functional size using either of these two standard measurement viewpoints, and the resulting sizes will obviously differ, since different functionality is revealed in each abstraction.

If, for example, our purpose were to support development effort estimation for some embedded avionics software, then we would almost certainly want to measure using the Direct User measurement viewpoint.

Alternatively, the total aircraft control system could be measured from the End User measurement viewpoint, where the end users are the aircrew. Such a measurement could be relevant in the context of some aircraft simulator training software, but it would be useless for the developer of any avionics software to be embedded in the real aircraft.

Introducing the concept of different standard measurement viewpoints has been a vital break-though for FSM. We now understand that 1G FSM methods were designed to measure size as seen by human End Users. It explains why 1G FSM methods cannot be easily applied to measure real-time software. For most practical purposes, the functionality of real-time software needs to be measured from the viewpoint of all of its ‘direct’ users.

COSMIC is also introducing into the method3 a factor that, if ignored, can result in highly misleading size measurements. This is the need to consider possibly varying ‘levels of granularity’ (or of decomposition) of the FUR to be measured. For example, in the early stage of a software project the development effort will probably need to be estimated when the requirements have only been agreed at a high-level of granularity, i.e. all the details have not yet been worked out. Guidance is therefore needed on how to scale a size measured at this high-level to the (bigger) size at the lower level of granularity of the full requirements.

Phase 2: Mapping to the COSMIC-FFP Conceptual Models

The COSMIC-FFP method relies on a number of carefully defined and inter-related concepts, which are summarized in Table 1. The Measurer’s task in this phase is, in effect, to extract a model of the software in terms of these COSMIC-FFP concepts from whatever software artefacts are available. Figure 1 presents the generic software model for COSMIC-FFP, in which these concepts are implemented.

Table 1: Principle Concepts of COSMIC-FFP

Phase 3: The Measurement Phase

In this phase we simply count the number of data movements identified in each functional process of the software to be measured, as the measure of its functional size. Each Entry, Exit, Read and Write is counted as one ‘COSMIC functional size unit’ or ‘Cfsu’.

Figure 1: The COSMIC-FFP Generic Software Model

Any one functional process must have a minimum size of 2 Cfsu, since to be meaningful it must have a ‘triggering Entry’ plus a successful outcome, either one Exit or one Write. However, there is no upper limit to the size of any one functional process (unlike the upper limit to the size of an IFPUG ‘elementary process’ which may not exceed 7 function points). In some individual military avionics software, we have measured single functional processes with over 100 data movements (i.e. of size > 100 Cfsu).

The method can also be used to measure the size of software changes for maintenance or enhancement activities. The size of a changed functional process is defined as the sum of the number of added, modified and deleted data movements.

What about data manipulation sub-processes?

All software includes some data manipulation sub-processes; why not identify and count them? The answer is simply that no-one agrees how to measure the functional size of an algorithm, so we cannot measure data manipulations on any scale that would be widely acceptable. We therefore make an approximation and assume that the functional size of an item of software is proportional to the number of its data movements, and assume each data movement includes certain associated data manipulation. For example, an Entry is defined to include all manipulations associated with input data validation that do not involve another data movement.

The COSMIC-FFP method is, therefore, considered applicable only for business applications and real-time software that is characterized as ‘movement-rich’. It cannot accommodate ‘algorithm-rich’ software involving heavy mathematical manipulations such as used in weather forecasting, stress analysis and such-like. And by the way, it is arguable that this omission is not so important. It is much more important and difficult to estimate the mathematical or engineering effort involved in developing an algorithm in the first place, essentially a creative activity, than to estimate the effort to encode it in software.

But what if the software you wish to measure is mostly ‘movement–rich’ but includes some significant, localized mathematical algorithms? The COSMIC-FFP method allows for a ‘local extension’ whereby an organization can define its own local scale for sizing algorithms alongside the Cfsu scale for sizing data movements. Alternatively, if your Purpose is estimating project effort, the measurement Scope can be defined to exclude the algorithms. The sizing and estimating process can then be applied only to the ‘movement-rich’ functionality whilst estimating for the algorithms can be dealt with by another appropriate process.

3. A Simple Example

Applying the COSMIC-FFP method to size real-time software in the Direct User measurement viewpoint is normally remarkably simple. Although today the measurement process is manual, there is reasonable hope for automation of the sizing if the FUR become available in a Computer Aided Software Engineering (CASE) tool [7].

Some simple, fully documented, ‘real’ case studies of real-time software are available from the COSMIC web site for free download [8]. For the purposes of this article, I will illustrate the measurement process with a simple example from my house, which has a basic intruder alarm system. Until I started this article, I had never considered how it might work and I have never developed any embedded software in my life. But having looked at it, I believe that a diagram showing the embedded software and its (direct) users must look something like the diagram presented in Figure 2 and that I can figure out how to measure the functionality.

The system has a conventional keypad and red/green LED’s for the human interface. The system is also connected to a device that senses whether the main front door is open or not, several internal movement detectors, and an internal and an external alarm. There is a battery to take over if the main power supply fails, so there must be a main voltage detector. Finally, our PIN code is stored in the system and can be changed, so there must be some persistent storage hardware, which, no doubt, also contains various system parameters.

There must be an internal clock mechanism, since certain functions must be completed within pre-set times. For example, having set the ‘exit code’ before leaving the house, the front door must be closed within a given time or alarms sound. I assume the internal clock is started in the software each time it is needed, so this functionality to keep track of elapsed times (by counting clock-cycles?) is a form of data manipulation, which we can ignore.

Figure 2: The Simple Intruder Alarm System

I have no idea whether the software periodically polls its input devices or whether the devices send signals to the software whenever they detect something. But this difference has no effect on how we map the software functionality to the COSMIC-FFP Generic Software Model (see Figure 1).

The following appear to me to be the unique types of events that the alarm can detect, each of which triggers a unique functional process. There are probably more event types, known only to the maintenance engineer.

• A new, or changed, PIN code is entered via the keypad (2 events)
• The ‘exit code’ is entered via the keypad thereby activating the system (before leaving and closing the front door within a pre-set time)
• The front door sensor detects that it is open whilst the alarm system is activated
• The system is activated, or de-activated, whilst in the house (2 events), e.g. when retiring upstairs to bed
• A movement detector signals a movement whilst the alarm system is activated
• The mains voltage detector signals main electrical failure, or its restoration (2 events)

Now let’s examine, as an example, the functional process that is triggered when I return to the house, with the alarm activated. When I unlock and open the front door, the internal alarm starts ‘beeping’, and I must then enter our PIN code within a pre-set time to de-activate the system and stop the internal alarm ringing. If I do not enter the PIN code before the pre-set time, or I enter the wrong code more than three times, the external alarm also starts ringing. This functional process must have the data movements identified in Table 2.

Table 2: Data movements within the 'Open door w/ alarm activated' Functional process

Note that the analysis of this functional process is very simple. I do not need to understand the detailed logic of, or the precise sequence of steps of, the process. The size measured accounts for all the data movements arising from all possible paths through the process; the size does not depend on the input values to the process which may determine any particular actual path.

The example illustrates that any software engineer with a clear understanding of the COSMIC-FFP concepts should be able to apply the measurement process to any real-time software requirements, or to the functionality of an existing software system with which they are familiar.

But this example may now raise another objection to the method. Applying the COSMIC-FFP method as described above requires a quite detailed statement of requirements. In a real project, you may well need to estimate a size well before you have the requirements in such detail, i.e. at a high level of granularity’, as mentioned above. The Measurement Manual [6] therefore gives guidance on how to construct your own local, approximation-variant of the method that can be used when the requirements are known only at some ‘higher level’, that is, in less detail. See [13] for an example of an approach to early sizing.

4. COSMIC-FFP for Estimating Project Effort and Duration

As of today, if you have mastered the functional sizing step (A), there are three possible approaches to the estimating step (B).

Approach 1: The first approach is to re-calibrate your current estimating method by substituting sizes measured in Cfsu for the sizes currently used (most likely estimates of SLOC?). This will require measuring the functional sizes of several existing software products for which the corresponding effort and duration are known. There is really no way of avoiding this step, but measuring some existing software is essential anyway to gain experience and build confidence in the sizing method. Several academic papers, e.g. [9] and [10], have been published showing results for this approach, including for maintenance and enhancement projects, using the COCOMO model and fuzzy logic. See [11] for an example of a tool to support this process.

Approach 2: If you have no established estimating method, then you can measure several existing software products (using COSMIC-FFP) from a domain where there is a high degree of project commonality (e.g. same technologies used, similar time/effort constraints, similar project team skill levels, similar risk, etc.) and then carry out your own statistical analyses so as to develop formulae to predict effort and duration from software size.

Approach 3: Finally, it is possible to measure the functional size of a new requirement set and then use ISBSG benchmark data to produce a first, ballpark estimate of effort. As of early 2006, there are data on around 95 projects measured using COSMIC-FFP in the ISBSG database and a report is available [12] from ISBSG with an analysis of the data. Of course, not many of these projects are yet from the real-time defence software community, but the number of projects submitted to the ISBSG database is growing and a special effort is being made in 2006 to gather more such data. This approach will become much more valuable in time, especially to provide sanity checks on early estimates.

5. COSMIC-FFP Today: Conclusions and

Lessons Learned

Over the five years since the first detailed account of the COSMIC-FFP method was published, a lot of practical experience has been gained and a lot has been learned.

The most gratifying aspect of this experience is that the underlying software models have been found to be robust when applied to an enormous variety of types of software. I believe the greatest lessons we have learned have been to recognise the importance of defining the measurement viewpoint, or abstraction, and the level of granularity to be used for a measurement. We have achieved our original goal of being able to measure credible functional sizes of real-time software for the first time, whilst continuing to be able to measure business application software.

The COSMIC-FFP method is a simple, proven, practical method of functional sizing for real-time software and is therefore a key ingredient for effort estimating methods and performance measurement. It is characterized as follows:

• It is stable and standardized. As we learn how to improve our explanation of the method, we continue to refine the Measurement Manual, but the underlying concepts are solid.
• The method is 100% ‘open’. The Measurement Manual is available in four languages and translations are scheduled or underway in another three.
• It has been very widely applied in both the business and real-time software domains. In the latter, I am personally aware of its successful application to size software in the fields of avionics, telecommunications, process control, paper copiers, digital cameras, vehicle engine management systems, and scientific instruments. Geographically, there is now substantial experience in Asia, Australasia, Europe and North America.
• Various informal tests have been carried out on the repeatability of measurements by different measurers working from the same specification. The results are both encouraging and predictable. If the specifications are clear, the measured sizes are highly repeatable. If there is ambiguity in the specifications, the results will not be repeatable.

One of our most intriguing and widespread findings is, namely, that applying the COSMIC-FFP sizing method is an excellent method for quality control of software requirements. If requirements cannot be sized, the software cannot be built without the developers having to make some assumptions. Furthermore, if you have a mapping of the functional requirements to the COSMIC-FFP conceptual models you have, in the catalogue of functional processes, an excellent foundation for requirements traceability and test case design.

The necessary infrastructure to support world-wide use of the method is now becoming well-established. This includes benchmark data, organizations that can provide training and consulting services, automated tools to support data collection and estimating (see [4] for more details), and certification examinations [8].

There is now a steady stream of papers at software metrics conferences demonstrating the growing interest by academics in research using COSMIC-FFP based measurements.

Looking Ahead

The future holds some interesting possibilities. Why not use the COSMIC-FFP method to size information processing functionality before the hardware/software allocation is decided? We can see no reason why not, but it needs testing on major systems. (It’s trivial to size the functionality of a pocket calculator without knowing if the functions are embedded in hardware or software.) Why not size software functional requirements automatically if they are held in a CASE tool? – Another topic already being explored [7].

If your organisation decides to adopt the COSMIC-FFP sizing method, please submit as much of your data as possible to the ISBSG database. It’s a real-chicken-and-egg challenge, which you can help to solve. The more benchmark data that is made available, the more valuable the method will become, the more data that will be generated, and so on. COSMIC has laid the foundations. Now the method’s users can add the greatest value.

And finally, remember it is an open method. Our documentation and support can always be improved. If you have suggestions or contributions for improvement, e-mail me at [email protected].

References

[1] A.J. Albrecht, ‘Measuring Application Development Productivity’, Proc. IBM Applications Development Symposium. GUIDE International and Share Inc., IBM Corp., Monterey, CA Oct 14-17, 1979, p83

[2] ‘Function Point Counting Practices Manual, Release 4.2’, International Function Point Users Group, 2004

[3] ISO/IEC 14143:1998 ‘Software Engineering – Software Measurement – Functional Size Measurement, Part 1: Definition of Concepts, International Organization for Standardization, Geneva, Switzerland, 1998.

[4] See www.cosmicon.com

[5] ISO/IEC 19761:2003, ‘Software Engineering- COSMIC-FFP – A Functional Size Measurement Method’, International Organization for Standardization, Geneva, Switzerland, 2003.

[6] The COSMIC Implementation Guide to ISO/IEC 19761:2003’ (the ‘Measurement Manual’), Version 2.2, obtainable from www.gelog.etsmtl.ca/cosmic-ffp.

[7] Azzouz, S.; Abran, A., (2004), ‘A Proposed Measurement Role in the Rational Unified Process (RUP) and Its Implementation with ISO 19761: COSMIC-FFP’, Proceedings of Software Measurement European Forum - SMEF 2004, Rome, Italy.

[8] See www.gelog.etsmtl.ca/cosmic-ffp

[9] Ali Idri, Alain Abran, ‘COCOMO Cost Model Using Fuzzy Logic’, 7th International Conference on Fuzzy Logic Theory and Technology, Atlantic City, NJ, February 27th 2000

[10] Gu Xunmei, Song Guoxin, Xiao Lizhong, ‘Estimating with COSMIC-FFP Functional Size Measurement Using Fuzzy Sets’, 3rd SMEF 2006, Rome, Italy, May 10 – 12th, 2006

[11] See http://www.telmaco.com/pdmetco15e.pdf

[12] ‘An Analysis of Software Projects Using COSMIC-FFP’, International Software Benchmarking Standards Group, January 2004

[13] Conte, M.; Iorio, T.; Meli, R.; Santillo, L., (2004), E&Q: An Early & Quick Approach to Functional Size Measurement Methods, Software Measurement European Forum (SMEF), Rome, Italy.

¹In ISO/IEC 19761:2003, there is a non-normative Note which states that ‘The Purpose and Scope of a specific FSM when combined are referred to as the ‘Viewpoint’. Subsequently, it was realized that ‘Measurement Viewpoint’ is a separate concept and that we should build upon the existing standard ISO/IEC 10746-2, which include the definitions of ‘abstraction’ and ‘viewpoint’

²In the current v2.2 of the Measurement Manual [6], this is known as the ‘Developer’ measurement viewpoint. The replacement, ‘Direct User’ name is introduced in v3.0 of the Measurement Manual, due to be published in late-2006

³See the next v3.0 of the Measurement Manual for details, to be published in late-2006

About the Author

Charles Symons is an independent consultant, leading COSMIC (the Common Software Measurement International Consortium) jointly with Professor Alain Abran of the E’cole de Technologie Supérieure, Université du Québec, Montréal, Canada.

He graduated in physics at Birmingham University and began his career with the UK Atomic Energy Authority. From there he moved to CERN (the European Centre for Nuclear Research) in Geneva, Switzerland, to Philips Electronics in the UK and the Netherlands, and to Xerox. After this 25-year ‘apprenticeship’ in most aspects of computing, he became a management consultant with Nolan Norton & Co and later KPMG in London, advising on IT strategy and on improving the performance of the IT function. He is now semi-retired, living in the UK and the French Alps, relying on the internet to stay in touch with the COSMIC network.

Author Contact Information Email: [[email protected]]