Overview
ISO/IEC 4944:2024 - Information technology - User interfaces - Evaluating usability of natural user interfaces (NUI) provides a structured framework, requirements and recommendations for evaluating the usability of natural user interfaces used in systems, products or services. The standard focuses on measuring and reporting usability in terms of effectiveness, efficiency and satisfaction, and includes guidance for assessing the naturalness of interactions (speech, gestures, touch, gaze and multimodal combinations). It can also be applied to other emerging user interfaces.
Key topics and requirements
- Evaluation framework: Defines a usability evaluation approach consistent with ISO/IEC 9241-11, emphasizing that usability depends on specified users, goals/tasks and contexts of use.
- Scope of evaluation: Covers both the usability of the UI for accomplishing tasks and the specialized evaluation of the naturalness of NUI interactions.
- Components to consider:
- Diverse users (abilities, accessibility needs)
- Diverse contexts (environmental, cultural, organizational)
- Goals, tasks and interaction sequences
- System/product/service as the evaluation target
- Measuring usability:
- Purpose-driven measures and reporting
- Effectiveness: task completion, accuracy of outcomes
- Efficiency: resources (time, effort) expended to complete tasks
- Satisfaction: user comfort, acceptance and perceived experience
- Additional measures for naturalness of interactions (how closely interactions align with everyday human behaviors)
- Evaluation methods: Emphasizes evaluation based on actual use and provides content elements and format guidance for reporting usability evaluation results.
Practical applications
Who uses ISO/IEC 4944:2024:
- UX researchers and usability practitioners evaluating NUIs
- Product designers and interaction designers building voice, gesture, touch or multimodal interfaces
- Developers and QA teams validating interaction naturalness and task performance
- Accessibility and user experience specialists ensuring diverse users and contexts are considered
- Project managers and procurement teams specifying usability requirements for NUI-enabled products and services
Practical uses:
- Designing test plans and metrics for NUI prototypes and production systems
- Comparing alternative interaction metaphors for naturalness and task performance
- Documenting and reporting usability findings for regulatory, procurement or design iteration purposes
- Extending evaluation approaches to other emerging interfaces
Related standards
- ISO 9241-11: Ergonomics of human-system interaction - Usability: Definitions and concepts (normative reference for usability concepts used in ISO/IEC 4944)
- ISO 9241-110 (referenced concepts such as interaction principles)
Keywords: ISO/IEC 4944:2024, natural user interfaces, NUI usability, usability evaluation, usability measurement, effectiveness efficiency satisfaction, naturalness, user interfaces, ISO 9241-11.