Overview
ISO/IEC 23007-2:2012 - Information technology - Rich media user interfaces - Part 2: Advanced user interaction (AUI) interfaces - specifies interoperable data formats and interface concepts to connect scene descriptions (video, audio, 2D graphics, animations) with advanced user interaction devices. The standard positions AUI as a bridge between remotely deployed scene compositions (e.g., Widgets, LASeR, BIFS) and system resources or sensors (multi‑touch, motion sensors, gaze trackers, g‑sensors). It supports semantic mapping from low‑level sensed data to higher‑level interaction patterns.
Key topics and technical scope
- AUI as bridge: Defines how scene descriptions access system resources via AUI interfaces or existing interfaces (for example DOM events).
- Interactivity patterns: Standardizes pattern containers and classification schemes, including:
- Geometric patterns (Point, Line, Rectangle, Arc, Circle) with reference coordinate system, syntax and semantics.
- Symbolic patterns - container formats and classification for well‑known symbolic gestures.
- Touch patterns - formats and basic touch classification (multi‑touch etc.).
- Hand posture and hand gesture patterns - formats to represent postures (grabbing, open palm) and gestures (waving).
- Schema and data formats: Guidance on schema wrapper conventions, basic data types and coordinate systems for consistent encoding of sensed events.
- Integration & examples: Syntax, semantics and examples for each pattern type to aid implementers.
- Supporting material: Informative annexes include relationship to MPEG‑V (ISO/IEC 23005), predefined message interfaces for a Widget Manager (Annex B), IDL interface definitions (Annex C) and patent statements (Annex D).
- Normative references: Uses W3C DOM Events, XML and XML Schema as foundational technologies.
Practical applications
- Enable portable, interoperable gesture and touch control for rich media applications (interactive TV, digital signage, VR/AR UIs).
- Support presentation engines and widget platforms to consume sensor-derived semantics (e.g., a detected circle becomes a “select” command).
- Facilitate device manufacturers and middleware vendors to expose standardized event/data formats for motion controllers, multi‑touch displays, gaze and posture sensors.
- Useful for content creators and third‑party scene composers who remotely deploy interactive scenes that rely on advanced input devices.
Who should use this standard
- UI/UX engineers and multimedia application developers
- Presentation engine and widget platform implementers
- Sensor and device manufacturers integrating advanced interaction inputs
- Standards architects and systems integrators building interoperable rich‑media ecosystems
Related standards
- ISO/IEC 23007 (MPEG‑U) - Part 1: Widgets; Part 3: Conformance and reference software
- ISO/IEC 23005 (MPEG‑V) - referenced in Annex A for device/semantic relationships
- W3C DOM Events, XML, XML Schema (normative references)
Keywords: ISO/IEC 23007-2:2012, Advanced User Interaction (AUI), rich media user interfaces, scene description, multi-touch, motion sensing, gesture recognition, geometric patterns, touch patterns.