Significance and Use
5.1 The primary purpose of this practice is to permit the user to validate numerical values produced by a multivariate, infrared or near-infrared laboratory or process (online or at-line) analyzer calibrated to measure a specific chemical concentration, chemical property, or physical property. The validated analyzer results are expected to be statistically indistinguishable, over diverse samples whose spectra are neither outliers or nearest neighbor inliers, to those produced by the primary test method to within control limits established by control charts for the prespecified statistical confidence level.
5.2 Procedures are described for verifying that the instrument, the model, and the analyzer system are stable and properly operating.
5.3 A multivariate analyzer system inherently utilizes a multivariate calibration model. In practice the model both implicitly and explicitly spans some subset of the population of all possible samples that could be in the complete multivariate sample space. The model is applicable only to samples that fall within the subset population used in the model construction. A sample measurement cannot be validated unless applicability is established. Applicability cannot be assumed.
5.3.1 Outlier detection methods are used to demonstrate applicability of the calibration model for the analysis of the process sample spectrum. The outlier detection limits are based on historical as well as theoretical criteria. The outlier detection methods are used to establish whether the results obtained by an analyzer are potentially valid. The validation procedures are based on mathematical test criteria that indicate whether the process sample spectrum is within the range spanned by the analyzer system calibration model. If the sample spectrum is an outlier, the analyzer result is invalid. If the sample spectrum is not an outlier, then the analyzer result is valid providing that all other requirements for validity are met. Additional, optional tests may be performed to determine if the process sample spectrum falls in a sparsely populated region of the multivariate space covered by the calibration set, too far from neighboring calibration spectra to ensure good interpolation. For example, such nearest neighbor tests are recommended if the calibration sample spectra are highly clustered.
5.3.2 This practice does not define mathematical criteria to determine from a spectroscopic measurement of a sample whether the sample, the model, or the instrument is the cause of an outlier measurement. Thus the operator who is measuring samples on a routine basis will find criteria in the outlier detection method to determine whether a sample measurement lies within the expected calibration space, but will not have specific information as to the cause of the outlier without additional testing.
Scope
1.1 This practice covers requirements for the validation of measurements made by laboratory or process (online or at-line) near- or mid-infrared analyzers, or both, used in the calculation of physical, chemical, or quality parameters (that is, properties) of liquid petroleum products and fuels. The properties are calculated from spectroscopic data using multivariate modeling methods. The requirements include verification of adequate instrument performance, verification of the applicability of the calibration model to the spectrum of the sample under test, and verification that the degree of agreement between the results calculated from the infrared measurements and the results produced by the PTM used for the development of the calibration model meets user-specified requirements. When there is adequate variation in property level, the statistical methodology of Practice D6708 is used to provide general validation of this equivalence over the complete operating range of the analyzer. For cases where there is inadequate property variation, methodology for level specific validation is used.
1.1.1 For some applications, the analyzer and PTM are applied to the same material. The application of the multivariate model to the analyzer output (spectrum) directly produces a PPTMR for the same material for which the spectrum was measured. The PPTMRs are compared to the PTMRs measured on the same materials to determine the degree of agreement.
1.1.2 For other applications, the material measured by the analyzer system is subjected to a consistent treatment prior to being analyzed by the PTM. The application of the multivariate model to the analyzer output (spectrum) produces a PPTMR for the treated material. The PPTMRs based on the analyzer outputs are compared to the PTMRs measured on the treated materials to determine the degree of agreement.
1.2 Performance Validation is conducted by calculating the precision and bias of the differences between results from the analyzer system (or subsystem) produced by application of the multivariate model, (such results are herein referred to as PPTMRs), versus the PTMRs for the same sample set. Results used in the calculation are for samples that are not used in the development of the multivariate model. The calculated precision and bias are statistically compared to user-specified requirements for the analyzer system application.
1.2.1 For analyzers used in product release or product quality certification applications, the precision and bias requirement for the degree of agreement are typically based on the site or published precision of the PTM.
Note 1: In most applications of this type, the PTM is the specification-cited test method.
1.2.2 This practice does not describe procedures for establishing precision and bias requirements for analyzer system applications. Such requirements must be based on the criticality of the results to the intended business application and on contractual and regulatory requirements. The user must establish precision and bias requirements prior to initiating the validation procedures described herein.
1.3 This practice does not cover procedures for establishing the calibration model (correlation) used by the analyzer. Calibration procedures are covered in Practices E1655 and references therein.
1.4 This practice is intended as a review for experienced persons. For novices, this practice will serve as an overview of techniques used to verify instrument performance, to verify model applicability to the spectrum of the sample under test, and to verify equivalence between the parameters calculated from the infrared measurement and the results of the primary test method measurement.
1.5 This practice teaches and recommends appropriate statistical tools, outlier detection methods, for determining whether the spectrum of the sample under test is a member of the population of spectra used for the analyzer calibration. The statistical tools are used to determine if the infrared measurement results in a valid property or parameter estimate.
1.6 The outlier detection methods do not define criteria to determine whether the sample or the instrument is the cause of an outlier measurement. Thus, the operator who is measuring samples on a routine basis will find criteria to determine that a spectral measurement lies outside the calibration, but will not have specific information on the cause of the outlier. This practice does suggest methods by which instrument performance tests can be used to indicate if the outlier methods are responding to changes in the instrument response.
1.7 This practice is not intended as a quantitative performance standard for the comparison of analyzers of different design.
1.8 Although this practice deals primarily with validation of infrared analyzers, the procedures and statistical tests described herein are also applicable to other types of analyzers which employ multivariate models.
1.9 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
