A synopsis of the Government Accountability Office (GAO), (formally the Government Accounting Office) report to congressional requesters titled "Defense Acquisition: Knowledge of Software Suppliers Needed to Manage Risks", (GAO-04-678), published in May 2004.

The Department of Defense (DOD) is concerned about the expansion of opportunities for exploiting vulnerabilities in defense weapon systems software that may result from increased reliance on prime contractors who, in turn, are outsourcing the development, implementing reuse, using COTS, and acquiring software. Additionally, contractors are growing through acquisitions, mergers, and a general trend toward globalization.

Concurrent with this software development paradigm shift, we are seeing increasing attempts by foreign entities to access U.S. technology and information, and countries and organizations hostile to the United States focusing on information warfare.

Do we know who is actually developing the software used in our weapons systems programs? Is there a significant risk resulting from the expansion of suppliers and the unknowns relating to the origins and security of the actual developers and the respective development environment? DOD thinks there is a risk that it needs to be identified and managed at the program level and that knowledge of all suppliers needs to be available for use in source selection.

Figure 1 depicts how supplier expansion is occurring and how it may encompass foreign involvement in the development process.

Figure 1. Scope of Supplier Expansion and Foreign Involvement

The spider-like image also conveys the complexity involved in identifying and tracking all suppliers and, specifically, sources of foreign involvement. The shaded oval identifies the current scope of control from the perspective of the Program Office. A solid, managed relationship exists between the prime contractor and the Program Office, but the remaining activity and information, which is primarily contractor driven, is essentially not visible to the Program Office.

To address DOD concerns, Congress asked the GAO to examine and report on DOD's efforts to identify and manage the risks associated with foreign involvement in software development in individual weapon systems programs.

For this study, conducted from April 2003 to May 2004, GAO selected 16 weapon systems varying in age and capability; reviewed relevant DOD guidance, policies, regulations and procedures; met with experts from the SEI and the weapons engineering community; and reviewed or solicited information from the Program Offices and their respective prime contractors. Appendix I of the GAO report provides further details of the scope and methodology of this study.

 

Summary of GAO Findings

GAO found that software security issues in general, and the risk associated with foreign involvement in particular, are taking a back seat to the main topics of focus on weapon systems programs - performance, cost and schedule. Reasons for this tend to fit into the following categories:

• Lack of policy to address the risk of foreign involvement

• Lack of communication among organizations who possess knowledge of foreign suppliers

• Lack of prioritization of software security relative to issues of cost, schedule, and performance

• Lack of clear accountability for addressing software security related risks

Figure 2 presents some of the quantifiable key findings with respect to the actions and viewpoints of the program officials. In general, program officials lacked awareness of foreign involvement, in either COTS, or their custom software. Consequently, they did not view any risk associated with foreign involvement as significant.

Figure 2. GAO Key Findings

They relied on the competence of their prime contractors to ensure quality and security and make good decisions about subcontractors, but few of the programs included specific software security requirements in their contractual agreements with the prime contractor. In the absence of specific requirements to address security risks associated with foreign involvement, the contractors are not dealing with it, electing instead to focus on meeting the specified contractual requirements.

Those programs that did identify software security as a risk focused on operational threats (e.g., limiting foreign access to software development facilities and denying foreign access to software code), not insider threats that might come from foreign involvement in software development.

GAO found that DOD policy and guidance is not currently addressing the issues of software development security and adopting a risk strategy for foreign involvement. Security policies for weapon systems software focus primarily on operational threats, not insider threats such as the insertion of malicious code by software developers. Additionally, security procedures that are in place tend to be applied after the software suppliers have been selected and, thus, do not provide the manager the opportunity to evaluate whether the risks associated with using a supplier are acceptable.

Some officials noted that acceptance testing for reused and COTS software limits its focus to proving functionality and, thus, closes the door to supplier information for those products.

GAO reported several situations in which knowledge of foreign involvement exists at some level, but is not routinely shared with the Program Office, either because there is no requirement to do so, or because the knowledge is acquired by other agencies relative to other functions, such as the export licensing process. Contractors request approval from the State Department, but the State Department does not automatically refer the application to DOD or the Program Office.

Some additional insights from the study are as follows:

• Although we know that practices like peer reviews and dedicated software testing can uncover malicious code and minimize defects, 50% of the programs made decisions about what code to test based on the risks and benefits to the functionality of the system, not on security. Experts agree that comprehensive testing (every line of code) to ensure complete security is perhaps physically impossible and would require immense resources.

• 75% of the programs reported using the Technology Assessment/Control Plan and/or the Program Protection Plan (documents that address the release of information to foreign governments through cooperative programs and military sales), but these documents do not provide specific information on suppliers who will be performing the work.

• 69% of the programs reported using the Defense Information Technology Security Certification and Accreditation Process (DITSCAP) to address general software security; however, this process does not govern contractors in cases where the requirements were not included in the original contract. In addition, the DITSCAP process bases its requirements on the program manager's assessment of risks. Therefore, unless the program manager has identified foreign software development as a risk, the process will not address it.

• Better software development practices alone (such as those represented by the SEI CMM levels of maturity) may reduce defects and improve overall software quality, but cannot be expected to address malicious software development activities intended to breach security.

• Program managers are encouraged (under the blanket of using sound systems engineering practices) to develop open software systems architectures, use COTS products, and make incremental improvements through code reuse. However, all of these practices have potential for introducing malicious code from unknown software development sources.

 

GAO Recommendations

The GAO concluded the DOD needs to take steps to ensure that software security is an integral element in decision-making and that program managers mitigate risks accordingly. They recommended that the Secretary of Defense take the following three actions to address risks attributable to software vulnerabilities and threats:

1. Require program managers (working with others as necessary) to define software security requirements, including identifying and managing software suppliers, and then communicate the requirements through the prime development contract to ensure that they are used in selecting suppliers

2. Require program managers to collect and maintain information on software suppliers (including software from foreign suppliers) and use the information to assess changes in supplier status and to adjust program security requirements

3. Require the Office of the Assistant Secretary of Defense for Networks and Information Integration (OASD-NII) and the Office of the Undersecretary of Defense for Acquisition Technology and Logistics (OUSD-ATL) to work with other organizations to ensure that weapons program risk assessments include attention to software development risks and threats.

DOD's concerns, in response to this report, are that the recommendations place too much responsibility on the program managers, and that insider threats are not limited to foreign suppliers. DOD believes that program managers should be able to rely on external resources to gain threat information on suppliers, and that formulation and oversight of security practices should be a collaborative function among several offices. Furthermore, a centralized information repository on software suppliers (including but not limited to foreign suppliers) is necessary because the cost of collecting and maintaining this information would require resources and assets beyond the scope of individual program managers. Perhaps identifying, tracking, and maintaining intelligence on security risks of software suppliers is best done at the DOD level so that it can be shared among the programs. Threat analysis, which drives the development of security requirements, should be carried out at the subsystem, system, and system-of-systems levels and not be limited to the scope, expertise, and resources of the individual program managers. [Source: GAO Report Appendix II, DOD Comments to the Recommendations]

Reading the actual report begs questions such as the following:

• Is the issue of malicious code potentially being inserted into a software component really a threat here and now?

• How far could someone get with this before it is discovered?

• Are foreigners the only people who could do this?

• What would it take to test for malicious code insertions (insider threats)?

• Whose job is it to verify that all the software used in a weapon system does not represent a security risk?

• What level of risk is acceptable (if any)? And at what cost?

• What do we need to know about software suppliers, or the software development environment, in order to be able to thwart such threats?

• How could we be sure that the information we collect on suppliers is, in fact, valid?

These and many more questions, collectively, hint at the complexity of the issue and its proposed solutions. Are GAO's recommendations simplistic given the realities of the issue? If implementing software security measures requires continuous tracking of all suppliers of all software products and results in enormous costs, who decides what the priority of security should be relative to overall cost, schedule and software capability requirements? Perhaps these questions have wetted your appetite for reading the report in its entirety. It is available for download at http://www.gao.gov/new.items/d04678.pdf

 

Author Contact Information

Ellen Walker

Data and Analysis Center for
Software (DACS)

Ph: 315-334-4936

E-mail: [email protected]