Overview
ISO/IEC 15938-18:2023 specifies conformance testing procedures and provides reference software for the compression of neural networks used in multimedia content description and analysis. This Part 18 companion standard supports implementations of ISO/IEC 15938-17 by supplying conformance bitstreams, test models, and encoder/decoder reference code intended to verify interoperability and correct implementation of the compression format.
Key topics and technical requirements
- Conformance testing framework
- Procedures to test both decoders and bitstreams (see Clauses 4.2–4.6).
- Test workflows for decoding self-contained bitstreams and bitstreams requiring out‑of‑band parameters.
- Use of provided test vectors and reference bitstreams hosted on the ISO website.
- Reference bitstreams and models
- A curated set of bitstreams and source models (e.g., MobileNetV2, VGG16, ResNet50, DCase, UC12B) to exercise features such as sparsification, pruning, topology pruning, partial data handling, and entropy coding variants.
- Test bitstreams demonstrate targeted features and encoder configurations to validate conformance.
- Reference software
- Includes both encoder and decoder implementations that follow ISO/IEC 15938-17 decoding/encoding behavior.
- Guidance on software location, licensing, installation and architecture (parameter reduction, approximation, reconstruction, encode/decode modules).
- Defined data structures and interfaces (examples: model_info, approx_data, nctm, nctm.nnr_model).
- Annex A provides an informative Python implementation to aid study and integration.
- Normative references
Practical applications and who should use it
- Developers and vendors of neural network compression tools and codecs - to validate encoder and decoder implementations against standard bitstreams.
- Manufacturers of multimedia devices and platforms - to ensure interoperable delivery and playback of compressed neural networks used in on‑device inference or metadata extraction.
- Conformance testing labs and QA teams - to run standardized tests and use reference bitstreams as test vectors.
- Researchers and integrators - to study compression methods, replicate reference results, and prototype implementations using the provided reference software (including Python examples).
Related standards (brief)
- ISO/IEC 15938-17:2022 - Compression of neural networks for multimedia content description and analysis (core format specification).
- ISO/IEC 21778 - The JSON data interchange syntax (referenced for metadata and interchange).
Keywords: ISO/IEC 15938-18:2023, compression of neural networks, conformance testing, reference software, bitstream, ISO/IEC 15938-17, multimedia content description, encoder, decoder, interoperability, reference bitstreams.