Significance and Use
5.1 The purpose of a ruggedness evaluation, or screening program, is to determine the sensitivity of the test method to changes in levels of pertinent operating factors using a small number of tests. Normally, operating conditions for a test method are defined along with allowable tolerances. A ruggedness analysis determines the effect of “worst-case” variation in operating conditions within the specified tolerances. If the ruggedness evaluation indicates that the factors have a statistically significant effect on test results, the method can be revised with smaller tolerances on operating conditions to reduce variation among test results.
5.2 This practice evaluates the effects of seven factors using eight testing conditions (treatments). The disadvantage of this approach is that it only estimates the main effects of the factors and does not detect the effects of interactions among factors. For this reason, this is a screening program and additional investigation is required to determine whether there are interaction effects.
5.3 A major reason for poor precision in test methods is the lack of adequate control over the sources of variation in testing procedures or testing environments. These sources of variation often are not controlled adequately because they were not identified during the development of the test procedures as having a large effect on test results. This practice provides a systematic procedure to establish the required degree of control for different testing parameters.
5.4 All new test methods must be subjected to an interlaboratory program to develop a precision and bias statement. These programs can be expensive and time-consuming, and the result may show that the method is too variable and should not be published without further revision. Interlaboratory studies may give the subcommittee an indication that the method is too variable, but they do not usually give a clear picture of the causes of the high variation. Application of this practice using one or two laboratories before finalizing the test method and conducting the interlaboratory study is an economical way to determine these causes.
5.5 Many existing test methods were developed before there was a requirement for precision and bias statements. Since this became a requirement, most of these test methods have developed precision and bias statements, and the result is that many have been found to suffer from relatively large amount of variation. This practice provides a relatively simple and economical way to investigate the causes of variation in test methods, so that a subcommittee will have some guidance as to which parts of the test method need to be revised.
5.6 The procedure can be used for a screening program within a single laboratory, but involvement of at least three laboratories is recommended, particularly if the single laboratory were to be the one that developed the test method. This is particularly important for new test methods. The originating laboratory is so much a part of the development of the test method that it is difficult for it to be objective in spotting any problems in the clarity of the test method instructions. Two additional laboratories will probably contribute fresh critical review of the validity of the test method and provide assistance in clarifying the instructions of the test method if needed. This practice, however, is not intended to provide information on multilaboratory precision, but it does provide some information on single-operator precision, which could be used to develop a temporary repeatability statement until the interlaboratory study is completed.
Scope
1.1 This practice covers a procedure for evaluating the ruggedness of a test method by determining the effects of different experimental factors on the variation of test results. The procedure is intended for use during the development of a test method before the interlaboratory study is executed, such as those described in Practices C802 and E691.
1.2 This practice covers, in general terms, techniques for planning, collecting data, and analyzing results from a few laboratories. Appendix X1 provides the details of the procedure with an example and Appendix X2 provides additional information on the methodology.
1.3 The practice is not intended to give information pertinent to estimating multilaboratory precision.
1.4 The system of units for this practice is not specified. Dimensional quantities in the practice are presented only in illustrations of calculation methods.
1.5 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.6 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
