— Developing Practitioner Assistance
Dr. Laura L. Pullum, Dr. Marjorie A. Darrah, and Mr. Brian J. Taylor, Institute for Scientific Research, Inc.
Neural networks represent a class of systems that do not fit into the current paradigms of software development and certification. Instead of being programmed, a learning algorithm “teaches” a neural network using a set of data. Often, because of the complex mathematical routines and nature of the training data, the neural network is considered a “black box” and its response may not be predictable.
In most instances, traditional testing techniques prove adequate for the acceptance of a neural network system. However, in more complex, safety- and mission-critical systems, the standard neural network training-testing approach is not able to provide a reliable method for certification. The use of artificial neural networks within NASA applications is expected to increase over the next few decades. Currently, there are over 20 NASA funded activities that use neural network technology, some of which are mission- and safety-critical. Verifying correct operation of neural networks within NASA projects, such as autonomous mission control agents and adaptive flight controllers, or within nuclear engineering applications, such as safety assessors and reactor controllers, requires a rigorous verification and validation (V&V) approach.
This V&V challenge is further compounded by adaptive neural network systems; ones that modify themselves, or “learn,” during operation. These systems continue to evolve during operation, for better or for worse. Traditional software assurance methods fail to account for systems that change after deployment. Furthermore, no overall standard exists that addresses a comprehensive V&V process specifically for neural networks. Although techniques do exist that apply to the V&V of neural networks, many are still underdeveloped or not sufficiently tested.
As the facility responsible for improving software safety, reliability, and quality of programs and missions, the
NASA Independent Verification &Validation (IV&V) Facility will be increasingly challenged to certify and evaluate software systems that contain neural network technologies. To prepare for this imminent need, the NASA IV&V Facility has funded an initiative for the IV&V of neural networks, the goal of which is to develop a new software assurance methodology specifically for neural networks.
The resulting methodology will incorporate state-of-the-art practices in the V&V of neural networks, along with the experiences and knowledge from work with intelligent flight control systems.
The methodology addresses each life cycle process and is designed so that IV&V practitioners may apply this methodology at various stages within
the project lifecycle.
The methodology will be written using the IEEE Standard for Software Verification and Validation, IEEE Std. 1012-1998 (IEEE 1012 1998) as a base document, with the intent of incorporating the methodology as a supplemental procedure. The methodology will provide guidance to the practitioner performing IV&V on neural network systems.
One of the first activities in this effort was the collection and evaluation of existing neural network V&V techniques. This resulted in a compilation and description of the state-of-the-art and -practice in V&V of neural networks, reported in (ISR 2002). A subsequent task involved conducting extensive research on V&V of neural network techniques that are more applicable to the independent V&V of neural networks, such as formal methods, stability analysis, run-time monitoring, testing, visualization, failures modes and effects analysis (FMEA), risk analysis, automated neural network selection, and neural network design verification. Some of this research is continuing, but the completed research is detailed in (ISR 2003a, Taylor 2003, Smith 2003, Darrah 2004). The IEEE 1012 standard for software V&V was concurrently reviewed, noting those tasks in which additional guidance would be required for application to neural network systems (ISR 2003b).
The next steps in the research for this effort will integrate new techniques with past experiences to develop practitioner guidance and associated training materials. To develop a methodology for the IV&V of neural networks we will incorporate the research findings on techniques to complement an accepted industry standard, IEEE 1012. To test the methodology, we will apply
it to the Intelligent Flight Control2 (IFC) system’s first generation flight control concept designed to identify aircraft stability and control characteristics using neural networks, and use this information to optimize aircraft
This project is performance in both normal and simulated failure conditions. Ultimately we will develop training materials to assist the practitioner in the use of the methodology.
Significant applied research has been conducted that addresses the V&V of neural networks. The results of NASA Dryden Flight Research Center (DFRC) and the NASA Ames Research Center (ARC) efforts, documented in “Verification and Validation of Neural Networks for Aerospace Systems” (Mackall 2002), focus on the V&V of pre-trained neural networks. The “V&V of Advanced Systems at NASA” (Nelson 2002) provides insight into neural network verification problems. The “Software Verification and Validation Plan for the Airborne Research Test System II Intelligent Flight Control Program” (ISR 2000) outlines some of the V&V processes that were performed on the IFC system (IFCS) neural networks to support that project.
| Area of Research | Techniques Under Investigation |
| Formal methods | Neural network rule extraction techniques, development of automated rule extraction tools |
| Stability analysis | Lyapunov and stochastic approaches to stability, theoretical approach to general learning theory |
| Run-time monitoring | Framework for assessing runtime monitors to protect the overall system from neural network failures |
| Testing | Automated Test Trajectory Generator to generate statistically related system data for neural network reliability assessment, sensitivity analysis |
| Visualization | Visualization techniques to assist in improving the understanding of neural network design, training, and testing (includes analysis of simulation of IFCS system which uses neural networks and visual knowledge representations) |
| Neural network V&V guidance to software standards | IEEE 1012, IEEE/ISO 12207 mapping to V&V of neural network research results to identify guidance gaps |
| FMEA and risk analysis | Investigations into applicability of applying software risk assessment and software FMEA to neural networks and additional neural network failure modes required. |
| Automated neural network selection | Investigations into appropriateness of neural network architectures/learning approaches to specific problem domains |
| Neural network design verification | Proper approaches to neural network learning, layer design, activation function selection |
The work introduced here combines current research and application results into a methodology that can be applied by an IV&V practitioner faced with the task of verifying and validating a system containing neural networks. This research effort includes the partnership of the Institute for Scientific Research, Inc. (ISR) scientists and engineers with researchers at West Virginia University and NASA IV&V, as well as association with researchers at NASA ARC and NASA DFRC through the IFCS program. To date, this is the largest working group dedicated to the V&V of neural network systems.
The table 1 presents information on the techniques developed to date and planned for investigation in 2004.
As stated earlier, the intention of this effort is to provide guidance in the evaluation of neural networks for the IV&V practitioner, whether at NASA, other government agencies, or in industry. The methodology will take into consideration each process, activity, and task that the IEEE 1012 uses for traditional software and provide additional guidance as required for neural network V&V. This guidance will be provided in the style in which (ISO/IEC 12207 1998) provides guidance.
For an example, refer to the IEEE 1012 (IEEE 1012 1998) excerpts provided below. For a neural network system, the guidance might include information such as that described below.
Guidance will be developed from the results of the research and will likely be more directed than the above example (from our early results). The guidance will be provided for each task, activity, and process within each life cycle phase.
The result of this effort will be a complete methodology for the IV&V of neural network systems that addresses the entire software life-cycle, is compatible with existing software standards, incorporates new technologies and methods research, and includes materials to train IV&V practitioners. This methodology will be made available to government and industry users of neural network technology with the intention that with the methodology, neural networks can be used more widely, verified and validated more completely, and used in more trusted and dependable systems.
Laura L. Pullum is a Principal Scientist at the Institute for Scientific Research, Inc., Fairmont, WV. For over 20 years, she has conducted research in software and system dependability, and holds a patent in this area. Dr. Pullum is the author of Software Fault Tolerance – Techniques and Implementation (2001) and has written over 350 additional papers and reports. She has served as Principal Investigator on efforts for the National Science Foundation, NASA, the U.S. Air Force, Navy, and Army, industry and universities. Dr. Pullum holds a Doctorate of Science in Systems Engineering and Operations Research, a M.S in Operations Research, an MBA, and a B.S. in Mathematics.
Marjorie A. Darrah is a Senior Scientist for the Institute for Scientific Research, Inc., Fairmont, WV. Her responsibilities at ISR include research and development in the areas of Neural Networks, Data Mining, and Virtual Reality. Dr. Darrah holds the position of Co-PI on “Development of Methodologies for IV&V of Neural Networks” and PI on “A Formal Method for Verification and Validation of Neural Network High Assurance Systems”, both NASA funded projects. Before joining ISR, she was the chairperson of the Division of Natural Sciences and a mathematics professor at Alderson-Broaddus College, Philippi, WV. Dr. Darrah holds a Doctorate, M.S, and B.S. in Mathematics.
Brian J. Taylor is a Senior Member of Research Staff for the Institute for Scientific Research, Inc. His work includes the development, analysis, and V&V of neural network components for F-15 adaptive flight control systems within the NASA DFRC Intelligent Flight Control project. He is also the Co-PI on a NASA IV&V Facility-funded effort for the “Development of Methodologies for IV&V of Neural Networks.” Taylor was involved in ISR’s first neural network research project where neural networks were used for analytical redundancy of sensors. This project, undertaken with West Virginia University, looked at Sensor Failure Detection, Identification, and Accommodation to improve fault-tolerant flight control and sensor reliability through use of neural networks. Taylor holds a M.S. in Electrical Engineering and B.S. in both Electrical Engineering and Computer Engineering.
Laura L. Pullum, Marjorie A. Darrah, Brian J. Taylor
Institute for Scientific Research, Inc.
320 Adams Street, P.O. Box 2720
Fairmont, WV 26555-2720
[email protected], [email protected], [email protected]
1 The research effort was funded through grant NAG5-12069 awarded by NASA Goddard Space Flight Center.
2 The IFC project is developing a real-time adaptable flight control system utilizing neural networks.
Darrah, Marjorie A., Brian J. Taylor and Spiro Skias. 2004. “Rule Extraction From Dynamic Cell Structure Neural Network Used in a Safety Critical Application.” In Proceedings of the 17th International FLAIRS Conference. Miami Beach, FL, 17-19 May 2004.
IEEE/EIA 12207.2-1997, IEEE/EIA Guide, Industry Implementation of International Standard ISO/IEC 12207:1995, (ISO/IEC 12207) Standard for Information Technology
–Software Life Cycle Processes
–Implementation Considerations,
New York, NY, 1998. (ISO/IEC 12207 1998).
Institute of Electrical and Electronics Engineering, Inc. (IEEE). Software Engineering
Standards Committee. 1998. IEEE Standard for Software Verification and Validation. New York, NY. (IEEE 1012 1998).
Institute for Scientific Research, Inc. (ISR). 2000. Software Verification and Validation Plan for the Airborne Research Test System II Intelligent Flight Control Program. IFC-SVVP-F001-UNCLASS-120100. December 2000
Institute for Scientific Research, Inc. (ISR). 2002. Toward Reliable Neural Network Software for the Development of Methodologies for Independent Verification and Validation of Neural Networks. IVVNN-LITREV-F001-UNCLASS-111202. November 2002.
Institute for Scientific Research, Inc. (ISR). 2003a. Introduction to Development of Methodologies for Independent Verification and Validation of Neural Networks. IVVNN-INT-F001-UNCLASS-021403. January 2003.
Institute for Scientific Research, Inc. (ISR). 2003b. Draft Guidance for the Independent Verification and Validation of Neural Networks. IVVNN-GUIDE-D001-UNCLASS-101603. October 2003.
Mackall, Dale, S. Nelson, and J. Schumman. 2002. Verification & Validation of Neural Networks for Aerospace Systems. NASA Ames Research Center.
Nelson, S., and C. Pecheur. 2002. V&V of Advanced Systems at NASA for Northrop
Grumman Corp. Produced for the Space Launch Initiative 2nd Generation RLV TA-5 IVHM Project. NASA Ames Research Center.
Smith, James. 2003. “Certification of On-Line Learning Neural Networks.” In Proceedings of ASC 2003. July 2003.
Taylor, Brian J., and Marjorie A. Darrah. 2003. “Verification and Validation of Neural Networks: A Sampling of Research in Progress.” In Proceedings of AeroSense. Orlando, FL, 21-25 April 2003.
 |
 |
 |