Overview
ISO/TR 9547:1988 - Programming language processors - Test methods - Guidelines for their development and acceptability - is an ISO Technical Report (type 3) that provides a methodology for designing, documenting and accepting test methods (test suites) to determine whether a programming language processor (compiler, interpreter, translator) exhibits the characteristics required by the International Standard for a given programming language. The report focuses on practical, repeatable test methods based on a reference set of test programs, automated test tools, and well‑defined test reports.
Key Topics and Requirements
- Scope and purpose: Guidance for creating test methods that assess conformance of a processor to a language standard; intended to be read alongside the language’s ISO standard.
- Definitions: Clear definitions of key terms such as configuration, processor, test, method of test, test program, test suite (jeu de test), test report, tools, and subset.
- Test-suite design:
- Modular, maintainable sets of test programs that are as independent as possible.
- Include both conforming and non‑conforming tests where the language standard specifies rejection.
- Tests should have a single objective where possible and be self‑checking (produce explicit success/failure output).
- Parametrized tests to accommodate different configurations.
- Automation and tools:
- Emphasis on automated execution, result collection, analysis and automated report generation to ensure reproducibility and efficiency.
- Test tools should be portable and, where practical, written in the target language.
- Test procedures and documentation:
- A defined global test plan, execution order (if required), classification and objectives for each test class, and a list of language features not covered by the suite.
- Test reports must record configuration, options, results, and language features that could not be tested.
- Practical constraints:
- Feasibility, economic considerations, portability across configurations, and maintenance/revision of the test suite.
- Special topics: Handling of language extensions, testing subsets, and objective acceptance criteria for test methods. The report references ISO/IEC guidance documents for testing practice.
Applications and Users
ISO/TR 9547 is useful for:
- Compiler and interpreter developers creating conformance test suites during development and maintenance.
- Test laboratories and certification bodies evaluating conformance of language processors.
- Standards committees and language designers preparing language standards and test method approval.
- Quality assurance teams seeking structured test plans for portability and compliance testing.
Keywords: ISO/TR 9547:1988, programming language processors, test methods, test suite, conformance testing, compiler testing, automated test tools, portability, test plan, acceptability.
Related Standards
- ISO/IEC Guide 2 (referenced for definitions of “test” and “test method”).
- Language‑specific ISO standards (to be used alongside ISO/TR 9547 when developing concrete test methods).