Significance and Use
5.1 The primary purpose of this practice is to permit the user to validate numerical values produced by a multivariate, infrared or near-infrared laboratory or process (online or at-line) analyzer calibrated to measure a specific chemical concentration, chemical property, or physical property. The validated analyzer results are expected to be equivalent, over diverse samples whose spectra are neither outliers or nearest neighbor inliers, to those produced by the primary test method to within control limits established by control charts for the prespecified statistical confidence level.
5.2 Procedures are described for verifying that the instrument, the model, and the analyzer system are stable and properly operating.
5.3 A multivariate analyzer system inherently utilizes a multivariate calibration model. In practice the model both implicitly and explicitly spans some subset of the population of all possible samples that could be in the complete multivariate sample space. The model is applicable only to samples that fall within the subset population used in the model construction. A sample measurement cannot be validated unless applicability is established. Applicability cannot be assumed.
5.3.1 Outlier detection methods are used to demonstrate applicability of the calibration model for the analysis of the process sample spectrum. The outlier detection limits are based on historical as well as theoretical criteria. The outlier detection methods are used to establish whether the results obtained by an analyzer are potentially valid. The validation procedures are based on mathematical test criteria that indicate whether the process sample spectrum is within the range spanned by the analyzer system calibration model. If the sample spectrum is an outlier, the analyzer result is invalid. If the sample spectrum is not an outlier, then the analyzer result is valid providing that all other requirements for validity are met. Additional, optional tests may be performed to determine if the process sample spectrum falls in a sparsely populated region of the multivariate space covered by the calibration set, too far from neighboring calibration spectra to ensure good interpolation. For example, such nearest neighbor tests are recommended if the calibration sample spectra are highly clustered.
5.3.2 This practice does not define mathematical criteria to determine from a spectroscopic measurement of a sample whether the sample, the model, or the instrument is the cause of an outlier measurement. Thus the operator who is measuring samples on a routine basis will find criteria in the outlier detection method to determine whether a sample measurement lies within the expected calibration space, but will not have specific information as to the cause of the outlier without additional testing.
Scope
1.1 This practice covers requirements for the validation of measurements made by laboratory or process (online or at-line) near- or mid-infrared analyzers, or both, used in the calculation of physical, chemical, or quality parameters (that is, properties) of liquid petroleum products. The properties are calculated from spectroscopic data using multivariate modeling methods. The requirements include verification of adequate instrument performance, verification of the applicability of the calibration model to the spectrum of the sample under test, and verification of equivalence between the result calculated from the infrared measurements and the result produced by the primary test method used for the development of the calibration model. When there is adequate variation in property level, the statistical methodology of Practice D6708 is used to provide general validation of this equivalence over the complete operating range of the analyzer. For cases where there is inadequate property variation, methodology for level specific validation is used.
1.2 Performance Validation is conducted by calculating the precision and bias of the differences between results from the analyzer system (or subsystem) produced by application of the multivariate model, (such results are herein referred to as Predicted Primary Test Method Results (PPTMRs)), versus the Primary Test Method Results (PTMRs) for the same sample set. Results used in the calculation are for samples that are not used in the development of the multivariate model. The calculated precision and bias are statistically compared to user-specified requirements for the analyzer system application.
1.2.1 For analyzers used in product release or product quality certification applications, the precision and bias requirement for the degree of agreement are typically based on the site or published precision of the Primary Test Method.
1.2.2 This practice does not describe procedures for establishing precision and bias requirements for analyzer system applications. Such requirements must be based on the criticality of the results to the intended business application and on contractual and regulatory requirements. The user must establish precision and bias requirements prior to initiating the validation procedures described herein.
1.3 This practice does not cover procedures for establishing the calibration model (correlation) used by the analyzer. Calibration procedures are covered in Practices E1655 and references therein.
1.4 This practice is intended as a review for experienced persons. For novices, this practice will serve as an overview of techniques used to verify instrument performance, to verify model applicability to the spectrum of the sample under test, and to verify equivalence between the parameters calculated from the infrared measurement and the results of the primary test method measurement.
1.5 This practice teaches and recommends appropriate statistical tools, outlier detection methods, for determining whether the spectrum of the sample under test is a member of the population of spectra used for the analyzer calibration. The statistical tools are used to determine if the infrared measurement results in a valid property or parameter estimate.
1.6 The outlier detection methods do not define criteria to determine whether the sample or the instrument is the cause of an outlier measurement. Thus, the operator who is measuring samples on a routine basis will find criteria to determine that a spectral measurement lies outside the calibration, but will not have specific information on the cause of the outlier. This practice does suggest methods by which instrument performance tests can be used to indicate if the outlier methods are responding to changes in the instrument response.
1.7 This practice is not intended as a quantitative performance standard for the comparison of analyzers of different design.
