Part 4 of BS ISO/IEC 19795 prescribes the methods for technology and scenario evaluations of multi-supplier biometric systems that use biometric data conforming to biometric data interchange format standards.
The multi-part biometric data interchange format standard, ISO/IEC 19794, has been developed to foster interoperable exchange of biometric data. By defining open containers for image, signal and feature data, and constraining some of the properties of the samples, the standards enhance interoperability by requiring implementers to be able to handle a restricted set of all possible biometric samples.
Examples of this are the template standards of ISO/IEC 19794-2 and ISO/IEC 19794-8 which embed compact processed data from fingerprint images. Only samples of the same format type (several of which can be defined in the same part of ISO/IEC 19794) are intended to be interchangeable.
This part of BS ISO/IEC 19795 defines tests to specifically address absolute performance, sufficiency, and interoperability available from biometric data formatted to comply with established standards, particularly those developed in the various parts of ISO/IEC 19794. However, because this part of ISO/IEC 19795 references interchange formats generically, by referencing only their black box generation and use, it also applies to other open standards. One consequence of this approach is that the success of a test is predicated on the correctness and appropriateness of lower-level data elements and values, i.e. conformance to the respective standards. Therefore, the approach here is to require conformance testing as an integral part of the test. This is achieved by referencing formal published conformance tests or profiles of standards. For instance, an interoperability test of the ISO/IEC 19794-5 face format might reference an application profile of its Token image, which in turn might rely on ISO/IEC 15444-1 (JPEG 2000 core coding system).
BS ISO/IEC 19795-4 specifies requirements needed to assess
- Performance available from samples formatted according to a standard interchange format (SIF)
- Performance available when samples formatted according to a SIF are exchanged
- Performance available from samples formatted according to a SIF, relative to proprietary data formats
- SIF interoperability, by quantifying cross-product performance relative to single-product performance
- Performance available from multi-sample and multimodal data formatted according to one or more SIFs
- Performance interoperability of biometric capture devices.
In addition, this part of ISO/IEC 19795
- Includes procedures for establishing an interoperable set of implementations,
- Defines procedures for testing interoperability with previously established sets of implementations, and
- Gives testing procedures for the measurement of interoperable performance.
It does not establish a conformance test for biometric data interchange formats, or provide test procedures for online data collection.
Contents of BS ISO/IEC 19795-4 include:
- Conformance
- Normative references
- Terms and definitions
- Goals
- Coverage
- Target, biometric and interoperable application
- Purpose
- Interoperability and sufficiency testing
- Metrics
- Figures of merit
- Measuring component failure
- Interoperability matrices
- Fixed operating point interoperability
- Reporting failure of sBDB generators
- Proprietary performance
- Conducting and structure of a test
- Sample data
- Acquisition
- Representative data
- Collection of ancillary data
- Corpus size
- Removal of subject-specific metadata
- Origin of samples
- Conformance
- Reporting
- Constraints on the sBDBs
- Optional encodings
- Optional encodings from profile standards
- Deviation from the base standard
- Data encapsulation
- Components
- Supplier recruitment
- Provision of samples to suppliers
- General requirements on software implementations
- Prevention and detection of gaming
- Remedial testing
- Survey of configurable parameters
- Interpretation of the interoperability matrix
- Determination of interoperable subsystems
- Identifying interoperable combinations of subsystems
- Acceptable numbers of interoperable subsystems
- Combinatorial search for maximum interoperability-classes
- Multiple interoperable subgroups
- Statistical stability of the test result
- Interoperability with previously certified products
- Decertification considerations
- Continuity of testing
- Interoperability with previously certified generators
- Treatment of systematic effects
- Retroactive exclusion from analysis
- Overall sufficiency
