Significance and Use
4.1 ASTM regulations require precision statements in all test methods in terms of repeatability and reproducibility. This practice may be used in obtaining the needed information as simply as possible. This information may then be used to prepare a precision statement in accordance with Practice E177. Knowledge of the test method precision is useful in commerce and in technical work when comparing test results against standard values (such as specification limits) or between data sources (different laboratories, instruments, etc.).
4.1.1 When a test method is applied to a large number of portions of a material that are as nearly alike as possible, the test results obtained will not all have the same value. A measure of the degree of agreement among these test results describes the precision of the test method for that material. Numerical measures of the variability between such test results provide inverse measures of the precision of the test method. Greater variability implies smaller (that is, poorer) precision and larger imprecision.
4.1.2 Repeatability and Reproducibility—These two terms deal with the variability of test results obtained under specified laboratory conditions and represent the two extremes of test method precision. Repeatability concerns the variability between independent test results obtained within a single laboratory in the shortest practical period of time by a single operator with a specific set of test apparatus using test specimens (or test units) taken at random from a single quantity of homogeneous material obtained or prepared for the ILS. Reproducibility deals with the variability between single test results obtained in different laboratories, each of which has applied the test method to test specimens (or test units) taken at random from a single quantity of homogeneous material obtained or prepared for the ILS.
4.1.2.1 Repeatability Conditions—The single-operator, single-set-of-apparatus requirement means that for a particular step in the measurement process the same combination of operator and apparatus is used for every test result and on every material. Thus, one operator may prepare the test specimens, a second measure the dimensions and a third measure the breaking force. “Shortest practical period of time” means that the test results, at least for one material, are obtained in a time not less than in normal testing and not so long as to permit significant changes in test material, equipment or environment.
4.1.2.2 Reproducibility Conditions—The factors that contribute to variability in a single laboratory, such as operator, equipment used, calibration of the equipment, and environment (for example, temperature, humidity, air pollution) will generally have different effects in other laboratories, and the variability among laboratories will be greater .
4.1.3 Precision is reported as a standard deviation, coefficient of variation (relative standard deviation), variance, or a precision limit (a data range indicating no statistically significant difference between test results).
4.1.4 This practice is designed only to estimate the precision of a test method. However, when accepted reference values are available for the property levels, the test result data obtained according to this practice may be used in estimating the bias of the test method. For a discussion of bias estimation and the relationships between precision, bias, and accuracy, see Practice E177.
4.2 Observations, Test Determinations and Test Results—A test method often has three distinct stages: the direct observation of dimensions or properties, the arithmetic combination of the observed values to obtain a test determination, and the arithmetic combination of a number of test determinations to obtain the test result of the test method.
4.2.1 In the simplest of test methods a single direct observation is both the test determination and the test result. For example, the test method may require the measurement of the length of a test specimen dimension, which then becomes the test result.
4.2.2 A test determination may involve a combination of two or more observations. For example, a test method may require the measurement of the mass and the volume of the test specimen, and then direct that the mass be divided by the volume to obtain the density of the specimen. The whole process of measuring the mass and the volume, and calculating the density, is a test determination.
4.2.2.1 If the test method specifies that only one test determination is to be made, then the test determination value is the test result of the test method. Some test methods require that several determinations be made and the values obtained be averaged or otherwise combined to obtain the test result of the test method. Averaging of several determinations is often used to reduce the effect of local variations of the property within the material.
4.2.2.2 In this practice, the term test determination is used both for the process and for the value obtained by the process, except when test determination value is needed for clarity.
4.2.3 The test result is the final reportable value of the test method. The precision of a test method is determined from test results, not from test determinations or observations.
4.2.3.1 The number of test results conducted by each laboratory on a material that is required for an interlaboratory study of a test method is specified in the protocol of that study.
4.2.4 Test Specimens and Test Units—In this practice a test unit is the total quantity of material needed for obtaining a test result as specified by the test method. The portion of the test unit needed for obtaining a single test determination is called a test specimen. Usually a separate test specimen is required for each test determination.
4.3 The procedures presented in this practice consist of three basic steps: planning the interlaboratory study, guiding the testing phase of the study, and analyzing the test result data.
4.3.1 The planning phase includes forming the ILS task group, the study design, selection and number of participating laboratories, selection of test materials, and writing the ILS protocol. A well-developed test method, including a ruggedness test to determine control of test method conditions, is essential.
Note 1: In this practice, the term test method is used both for the actual measurement process and for the written description of the process, while the term protocol is used for the directions given to the laboratories for conducting the ILS.
4.3.2 The testing phase includes material preparation and distribution, liaison with the participating laboratories, and handling of test result data received from the laboratories.
4.3.3 The data analysis utilizes tabular, graphical, and statistical diagnostic tools for evaluating the consistency of the data so that unusual values may be detected and investigated, and also includes the calculation of the numerical measures of precision of the test method pertaining to repeatability and reproducibility.
4.4 The information in this practice is arranged as follows:
|
| Section |
| Scope | 1 |
| Referenced Documents | 2 |
| Terminology | 3 |
| Significance and Use | 4 |
|
|
|
| Planning the Interlaboratory Study (ILS) | Section |
| ILS Membership | 5 |
| Basic Design | 6 |
| Test Method | 7 |
| Laboratories | 8 |
| Materials | 9 |
| Number of Test Results per Material | 10 |
| Protocol | 11 |
|
|
|
| Conducting the Testing Phase of the ILS | Section |
| Pilot Run | 12 |
| Full Scale Run | 13 |
|
|
|
| Calculation and Display of Statistics | Section |
| Calculation of the Statistics | 14 |
| Tabular and Graphical Display of Statistics | 15 |
|
|
|
| Data Consistency | Section |
| Flagging Inconsistent Results | 16 |
| Investigation | 17 |
| Task Group Actions | 18 |
| Glucose ILS Consistency | 19 |
|
|
|
| Precision Statement Information | Section |
| Repeatability and Reproducibility | 20 |
|
|
|
| Tables | Table |
| Glucose in Serum Example | 1–4, 6–8 |
| Critical Values of Consistency Statistics, h and k | 5 |
|
|
|
| Figures | Figure |
| Glucose in Serum Example | 1–3 |
|
|
|
| Appendixes | Appendix |
| Theoretical Considerations | Appendix X1 |
| Pentosans in Pulp Example | Appendix X2 |
| Spreadsheet for E691 Calculations | Appendix X3 |
Scope
1.1 This practice describes the techniques for planning, conducting, analyzing, and treating the results of an interlaboratory study (ILS) of a test method. The statistical techniques described in this practice provide adequate information for formulating the precision statement of a test method.
1.2 This practice does not concern itself with the development of test methods but rather with gathering the information needed for a test method precision statement after the development stage has been successfully completed. The data obtained in the interlaboratory study may indicate, however, that further effort is needed to improve the test method.
1.3 Since the primary purpose of this practice is the development of the information needed for a precision statement, the experimental design in this practice may not be optimum for evaluating materials, apparatus, or individual laboratories.
1.4 Field of Application—This practice is concerned exclusively with test methods which yield a single numerical figure as the test result, although the single figure may be the outcome of a calculation from a set of measurements.
1.4.1 This practice does not cover methods in which the measurement is a categorization; however, for many practical purposes categorical outcomes can be scored, such as zero-one scoring for binary measurements or as integers, ranks for example, for well-ordered categories and then the test result can be defined as an average, or other summary statistic, of several individual scores.
1.5 This standard may involve hazardous materials, operations, and equipment. This standard does not purport to address all of the safety problems associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
