Some of the most significant changes that have confronted DoD software acquisition efforts in the past few years are the result of using Commercial Off-The-Shelf (COTS) software. However, these changes are not unique to the DoD, virtually all segments of US Government and industry have been forced to deal with the implications of COTS software. These changes are the inevitable and irreversible consequence of increasing industrial and social reliance on computing technology. And if this assertion is not convincing to the DoD program manager, there is a range of Government and DoD acquisition policies, guidelines, and directives that provide more than ample motivation for using COTS software.
The implications of COTS software on DoD software acquisition are many and varied, as suggested by the SEI monograph series on COTS software [1]. This short article is focused more narrowly on the topic of COTS software on software architecture. To side step the issue of what is meant by “architecture,” this article examines how COTS software affects the strategies and tactics employed by the successful system architect or lead designer. Although this article focuses on the architect, DoD program managers and executives will find this information useful in understanding the issues faced by integration contractors, and in assessing how well integration contractors are responding to these issues.
There are two common mistakes made by designers unfamiliar with the implications of using COTS software. The first mistake is to design a system without reference to COTS software on the assumption that products are merely an implementation detail to be filled in after the major design decisions have been made. The second mistake is the complement of the first, that is, to blithely allow one or more COTS products to dictate the design of a system. The consequence of the first mistake is that opportunities to use COTS products will be missed, while the consequence of the second mistake is vendor lock. The successful architect will have a more balanced approach.
It is important to understand that COTS products often have an unavoidable impact on system design. Consider, for example, an architectural trade-off analysis conducted by the SEI for the Federal Aviation Administration (FAA) [2]. This study analyzed the architectural implications of using two different commercial technologies for interprocess communication, CORBA and POSIX.21*. The result of the study showed that each technology imposed its own unique constraints on the system design, resulting in different system structures and quality attributes (e.g., modifiability and performance).
Tradeoffs involving COTS software are not limited to just the system architecture. For example, the CORBA design might have better modifiability than the POSIX.21 design due to its object-oriented nature, but in exchange might have worse performance. Making this tradeoff might involve requirements. Can performance requirements be relaxed to obtain benefits in modifiability?
Figure 1: Tradeoff Regions
Figure 1 illustrates tradeoff regions as intersections among COTS software, design, and requirements. The architect needs to be actively engaged in each of these regions. The CORBA-induced performance versus modifiability tradeoff lies at the intersection of COTS product, design and requirements. Other tradeoff decisions might involve only COTS products and the design. It is also possible for COTS products to influence system requirements independent of a design.
It might appear that it is difficult to manage all of these simultaneous tradeoffs and it is. Fortunately, there are tools that an architect can use to come to terms with these tradeoffs. The Architecture Tradeoff Analysis Methodology (ATAM) [3] used in the FAA study cited above is one tool. Another tool more closely linked to COTS software is formative evaluation. In brief, formative evaluation exposes how products ought to be used in a system. The architect will find formative evaluation useful early in the design process when there is more latitude for adapting design and requirements to product capabilities (and liabilities). A more detailed discussion of formative (and normative) evaluation can be found in an on-line tutorial on COTS software evaluation [4].
An ironclad rule of the software marketplace is that things change, and often change very quickly. New products and technologies emerge at a rapid pace (two thousand software products per month, according to CIO magazine [5]), and new versions of existing products add and modify capabilities in response to market pressures. The implication is clear: a design that has been influenced by commercial software may become infeasible in response to changes to that software. Of course, this takes a negative view of marketplace change when, in fact, it is precisely this market dynamism that produces steady improvements in product capabilities. So, in the same way that marketplace changes can render some design options infeasible, new design options may become feasible and perhaps desirable.
System designers have always had to accommodate the exigencies of change in the design activity. Designs evolve as more is learned about the problem at hand; and of course, changing requirements is the norm, not the exception. However, the software marketplace adds a new dimension of instability that becomes noticeably pronounced as the number of products used in a system increases. This instability is exaggerated where new and rapidly evolving technologies are employed, as is the case with Web technologies and distributed object technologies such as CORBA and JavaTM. It is an unfortunate “Catch 22” that these unstable technologies are usually the ones whose use is perceived (by customers, designers and end users) as being highly desirable, since it is precisely this interest that leads to the technology instability in the first place.
What can the architect do in the face of marketplace instability? One technique is to keep product-sensitive design options open for as long as possible. For large projects where the design activity may span many months this kind of “late binding” strategy may be appropriate. For example, three design options could be pursued: a safe option that is known to work with today’s products; an anticipatory option that is expected to work with new capabilities that have been announced but not yet shipped by vendors; and a “blue sky” option that is more aggressively futuristic. For projects with a tighter time frame for the design activity, an alternative to late binding is early binding, sometimes referred to as “anchor first.” In this strategy early design commitments are made on key products that are presumed to be stable. For example, integration infrastructures such as message-oriented middleware products, or tool suites from relational database vendors, are often used as design anchors.
Selecting products as design anchors has the appeal of simplifying the design process, but on the other hand increases the risk of a system becoming too dependent on a particular software vendor. What if the anchor (or its vendor) turns out to be less stable than anticipated? This issue is taken up in the implications of vendor lock.
The first two implications combine to establish a third implication: deep product expertise is a critical design asset. Simply put, good design decisions about software products can not be made in the absence of sufficient (and often deep) knowledge about those products. As the number of products used in the system increases, so also increases the need for spanning expertise knowledge of how ensembles of products (or technologies) can be integrated. Unfortunately, given the dynamism of the commercial software marketplace, product expertise is a wasting asset: the useful half-life of expertise in some key technology areas is surprisingly short. For example, until recently “thin clients” via JavaTM Applets was the “hot” technology. Today, experts in Web ensembles are much more skeptical about the universality of thin clients (see [6] for a good discussion of the pros and cons of thin clients and some practical alternatives).
Unfortunately, individuals with deep, spanning expertise across the range of products typically used in COTS-integrated systems for example, Web, database, transaction, distributed object, system management, and security technologies are exceedingly rare. Keeping current with any one product in any of these product areas is difficult enough; and tracking an entire category of products (Web products, for example) can be a full time job. It is an expensive proposition to develop and sustain this level of technology competency. How can the architect obtain the kind of product and technology expertise at the time it is needed? One choice frequently adopted is to hire consultants. In some cases this is the most economical approach, although there are two risks. First, the hiring organization often lacks enough expertise to assess the competency of the consultant and the consequence of accepting advice from charlatans is predictable. Second, consultants often have ulterior motives, especially if they are hired from a software product vendor, or if their expertise is limited in range rather than spanning if the consultant only knows how to wield a hammer, everything looks like a nail.
The architect does have another option in the use of model problems (a kind of formative evaluation technique see [4]) to develop “just in time” technology competency. Model problems are small-scale prototypes that are focused narrowly on specific critical design issues. Software products can be used to develop one or more model solutions, each representing design alternatives that have been proven feasible (or infeasible). The trick in this case is to have sufficient technology competency to recognize a critical design issue relating to the use of COTS software (integration issues are a good place to start) but this is not too much to expect from a COTS-savvy architect.
The software market is driven by differentiation, not standardization, and it is often innovative (i.e., non-standard) features that cinch software sales. The temptation to take full advantage of unique product features is understandable, especially in high-end, expensive products. Using vendor-specific features can provide enhanced system capabilities, but on the other hand makes the sustainability of the system dependent upon a single supplier. There is a complementary temptation to insulate systems from specific products, usually as a hedge against market dynamism. For example, if a vendor goes out of business a new product can be inserted in place of the old, and clients will not be affected. Insulating products provides stability, but an abstract interface that can be mapped to competing products forces the system to rely on the common subset of features found in products.
Although there is no universal answer to this tradeoff, it is important that the architect is aware of the conditions in which unanticipated and de facto vendor lock can arise. One simple technique is to ensure that every product used in a system has a viable competitor that is commercially available. If competitors exist, then a separate design decision about whether or not to insulate the design from the product (through abstract interfaces, for example) can be made. If there are no viable competitors, however, no amount of insulation can hide the reality of vendor lock. Note that “standards” are not completely sanative; in some cases vendors extend standards (SQL is a classic example), while in other cases too few products may implement a standard to prevent de facto lock-in.
Another technique for the designer to avoid vendor lock is to allow the use of product-specific features, but only for non-critical or discretionary parts of a system capabilities that can be sacrificed. This can be a good compromise, but it does introduce a very slippery slope.
The previous implications are combined into one last implication of great significance: the skills of the system architect must encompass both technical competency and business competency. This can be seen in each of the above implications:
Managing tradeoffs between commercial products and requirements requires deep knowledge of the products as well as the mission or business area being automated. Frequently, the architect will be required to negotiate directly with customers, explaining the business implications of using commercial software in addition to technical implications for example, impacts on cost to sustain the system, added operational efficiencies, and return on investment.
With respect to system and design instability, the architect must make early decisions regarding the kinds of technologies to be used. Clearly, some technologies are less stable than others; a decision to use a newer but less stable technology in place of an established but more stable technology requires investment analysis. Does mission criticality justify the added cost and risk of using unstable but “feature-rich” technologies? Or would “good enough” technologies suffice?
Concerning technology competency, the architect must understand the costs of acquiring product expertise versus the cost of making a poor decision. In some cases sufficient design risk will warrant a significant investment in acquiring competency perhaps a small-scale prototype will need to be developed. The point is that uncertainty is inevitable with COTS software, and the management of uncertainty invariably requires business analysis.
The need for business acumen is most apparent in addressing issues of vendor lock. Lock-in might be the outcome of a strategic alliance between product suppliers and integrators. Such alliances might bring competitive advantages to integrators, and cost and capability benefits to acquirers. Of course, technical and business risks accrue as well. In the DoD these issues can arise on a per-acquisition basis, given the size and longevity of the systems concerned. In these cases, the architect must play a key role in mediating the technical and business tradeoffs.
The system architect of the future will possess a range of personal and technical skills especially adapted to the implications of extensive use of commercial software components. Acquirers will need to understand these implications in order to be informed consumers. In the final analysis, building COTS-integrated systems requires a partnership of owner, integrator and product suppliers.
About the AuthorMr. Wallnau is a senior member of the technical staff at the Software Engineering Institute (SEI), Carnegie Mellon University, Pittsburgh, PA, where he co-leads the Commercial Off-The-Shelf (COTS) COTS-Based Systems project. Mr. Wallnau's current interests include the role of product evaluation in the system design activity, and the transition from COTS-based to component-based systems. Mr. Wallnau has published several papers and tutorials on COTS product and technology evaluation as well as on various facets of component-based software engineering with distributed objects. Prior to the SEI, Mr. Wallnau was the Unisys System Architect of CARDS, a $7M/Year DoD program focused on the use of COTS software as a strategy for improving software reuse. Mr. Wallnau graduated in 1985 summa cum laude from Villanova University with a B.S. in computer science. |
Contact InformationKurt C. WallnauSoftware Engineering Institute (412) 268-3265, Fax: (412) 268-5758 [email protected]  http://www.sei.cmu.edu |
 |
 |