|
|
| Software Testing by Statistical Methods
Mission Investigation and development of new methods for software testing based on stochastic processes and
statistical measures in order to improve the quality of software and to provide qualitative measures for
determining the probability that software correctly adheres to its specification. During 1997, ITL staff from the Statistical Engineering Division (898) and the Software and Systems Division (897) proposed a collaborative effort to research the potential and
practicality for applying statistical methods to software testing. This effort evolved into a new
ITL Competency proposal, entitled Software Testing by Statistical Methods, that was approved by the
NIST Director for initial funding in fiscal year 1998. The Software Testing by Statistical Methods project includes both white-box testing and black-box
testing to determine if a program conforms to its functional specifications. White-box testing assumes
that program source code is available for inspection and metering, whereas black-box testing allows
access to a program only through its defined interface. Functional specifications may be specified
using either conventional techniques or by formal methods. Conventional techniques usually specify formal
syntax with semantic rules written in natural language, whereas formal methods require that both syntax
and semantics be specified using mathematically rigorous techniques. In any case, the overall goal of
this project is to integrate statistical techniques into software testing to ensure software quality and
to provide quantitative measures of stability, reliability, and conformance to specifications. Initial priority for this project is on statistical methods potentially applicable to software defined
using conventional techniques. A companion project in the
Software Quality Group places more emphasis on
formal methods. Conventional specifications are important because a large majority of National, International,
and Internet standards are specified via non-formal methods, and a major mission of ITL is to provide conformance
testing techniques for such software standards.
Name and Email NIST Directory
Component Integration Testing - Working with a group of Computer Graphics vendors and users to define
usage models for estimating software reliability. Others - We would enjoy working with your company or industry group on any aspect of applying
statistical methods to software testing, possibly under a cooperative research and development agreement
(CRDA). For information contact any project participant. Project Proposal - Software Testing by Statistical Methods, July 10, 1997. Initial Tasks - Software Testing by Statistical Methods, August 22, 1997.
(HTML file) NISTIR 6129. David Banks, William Dashiell, Leonard Gallagher, Charles Hagwood, Raghu Kacker,
Lynne Rosenthal; Software Testing by Statistical Methods - Preliminary Success Estimates for
Approaches Based on Binomial Models, Coverage Designs, Mutation Testing and Usage Models,
March 12, 1998. James Yen, David Banks, P. Black, L.J. Gallagher, C.R. Hagwood, R.N. Kacker, L.S.Rosenthal;
Software Testing: Protocol Comparison, March 28, 1998, Eleventh International Software Quality
Week (QW' 98) conference, San Francisco. Charles Hagwood, Lynne Rosenthal, Raghu Kacker, Leonard Gallagher, James Yen; Reliability
of Conformance Tests, August 20, 1998. Proceedings of 22nd Annual International Computer Software
and Applications (COMSAC '98) conference, Vienna, Austria. Len Gallagher; Conformance Testing of Object-Oriented Components Specified by State/Transition
Classes, April 6, 1999, Draft technical report. |