ISO/TS 8000-81:2021 PDF

Data quality — Part 81: Data quality assessment: Profiling

Название (английский):

ISO/TS 8000-81:2021

Data quality — Part 81: Data quality assessment: Profiling

Статус документа:: Действующий
Формат:: Электронный (PDF)
Количество страниц:: 11
Дата публикации:: 20 мая 2021 г.
Издание:: ISO TS 8000 edition 1 version 1
ICS:: 25.040.40

Тип поставки:

Full version165 €Read online 24h10 €Read online 72h15 €

Язык документа:

ruen

Срок поставки:2 раб. дн. (перевод)

Цена:165 €

Описание (английский):

This document specifies a procedure for data profiling to generate the foundation for performing data quality assessment. This profiling is applicable to data sets that are either originally in a structure of tables and columns or are the output from a transformation to create such a structure. NOTE 1 Data profiling is applicable to all types of database technology. The following are within the scope of this document: — performing structure analysis to determine data element concepts; — performing column analysis to identify relevant data elements, including statistics about a data set; — performing relationship analysis to identify dependencies in a data set. The following are outside the scope of this document: — methods for extracting and sampling data to be profiled from a data set; — deriving data rules; — measuring the extent of nonconformities in a data set. NOTE 2 ISO 8000‑8 specifies approaches to measuring data and information quality. This document can be used in conjunction with, or independently of, quality management systems standards.

Технические детали

Технический комитет: ISO/TC 184/SC 4 - Industrial data
SKU: ISO/TS 8000-81:2021

Abstract

Overview

ISO/TS 8000-81:2021 - "Data quality - Part 81: Data quality assessment: Profiling" defines a standardized procedure for data profiling as the foundation for data quality assessment. The technical specification applies to data sets organized as tables and columns (or produced by transformations into that form) across all database technologies. The aim is to produce a data profile that helps organizations identify data quality improvement opportunities and support subsequent rule creation and governance.

Key topics and requirements

Scope of profiling
- Structure analysis, column analysis and relationship analysis are the three mandatory processes.
- Applicable to tabular data and outputs of transformations into tables/columns.
Structure analysis
- Inputs: data set and optional column metadata (names, descriptions).
- Output: a data element concept that captures the conceptual domain for subsequent analysis.
Column analysis
- Inputs: data set + data element concept.
- Activities: extract data elements, compare elements to actual values, determine the value domain.
- Outputs: constraints of value domain including cardinalities (row counts, distinct counts, nulls), storage characteristics (data types, lengths, decimals), and valid-value definitions (lists, ranges, patterns). Methods include discovery, assertion testing and visual inspection; automation tools can assist.
Relationship analysis
- Inputs: data set + data elements from column analysis.
- Activities: identify dependencies and correspondence between data structure and real-world items (requires collaboration with domain experts).
- Outputs: list of dependencies - primary/foreign keys, functional dependencies, derived columns, and synonym relationships (redundant or domain synonyms).
Explicit exclusions
- The specification does not cover data extraction/sampling methods, deriving data rules, or measuring the extent of nonconformities.

Practical applications

Establishing a repeatable data profiling baseline for data governance, data quality management, and data stewardship.
Informing data migration, master data management (MDM), analytics, BI, and regulatory compliance efforts by revealing structural issues, value-domain constraints and inter-column dependencies.
Supporting the design of validation rules and remediation plans (profiling provides inputs for rule derivation, though rule creation itself is outside the spec).
Useful for automated tooling adoption: selection, configuration and evaluation of data profiling tools.

Who should use this standard

Data quality managers, data stewards, data engineers, DBAs, business analysts, BI teams, auditors and architects responsible for data governance, MDM, ETL/ELT and analytics.

Related standards

ISO 8000 series (context for data quality)
ISO 8000-2 (Vocabulary)
ISO 8000-8 (Measuring data and information quality)
ISO 8000-61 and ISO/TS 8000-1 (data quality management and series overview)

Keywords: ISO/TS 8000-81, data profiling, data quality assessment, value domain, column analysis, relationship analysis, data governance, data stewardship.

Overview

Key topics and requirements

Scope of profiling
- Structure analysis, column analysis and relationship analysis are the three mandatory processes.
- Applicable to tabular data and outputs of transformations into tables/columns.
Structure analysis
- Inputs: data set and optional column metadata (names, descriptions).
- Output: a data element concept that captures the conceptual domain for subsequent analysis.
Column analysis
- Inputs: data set + data element concept.
- Activities: extract data elements, compare elements to actual values, determine the value domain.
- Outputs: constraints of value domain including cardinalities (row counts, distinct counts, nulls), storage characteristics (data types, lengths, decimals), and valid-value definitions (lists, ranges, patterns). Methods include discovery, assertion testing and visual inspection; automation tools can assist.
Relationship analysis
- Inputs: data set + data elements from column analysis.
- Activities: identify dependencies and correspondence between data structure and real-world items (requires collaboration with domain experts).
- Outputs: list of dependencies - primary/foreign keys, functional dependencies, derived columns, and synonym relationships (redundant or domain synonyms).
Explicit exclusions
- The specification does not cover data extraction/sampling methods, deriving data rules, or measuring the extent of nonconformities.

Practical applications

Establishing a repeatable data profiling baseline for data governance, data quality management, and data stewardship.
Informing data migration, master data management (MDM), analytics, BI, and regulatory compliance efforts by revealing structural issues, value-domain constraints and inter-column dependencies.
Supporting the design of validation rules and remediation plans (profiling provides inputs for rule derivation, though rule creation itself is outside the spec).
Useful for automated tooling adoption: selection, configuration and evaluation of data profiling tools.

Who should use this standard

Data quality managers, data stewards, data engineers, DBAs, business analysts, BI teams, auditors and architects responsible for data governance, MDM, ETL/ELT and analytics.

Related standards

ISO 8000 series (context for data quality)
ISO 8000-2 (Vocabulary)
ISO 8000-8 (Measuring data and information quality)
ISO 8000-61 and ISO/TS 8000-1 (data quality management and series overview)

Keywords: ISO/TS 8000-81, data profiling, data quality assessment, value domain, column analysis, relationship analysis, data governance, data stewardship.

ISO/TS 8000-81:2021 PDF

Технические детали

Abstract

Overview

Key topics and requirements

Practical applications

Who should use this standard

Related standards

Похожие стандарты

Abstract

Overview

Key topics and requirements

Practical applications

Who should use this standard

Related standards

Похожие стандарты

Data quality — Part 51: Data governance: Exchange of data policy statements

Data quality — Part 2: Vocabulary — Amendment 1

Data quality — Part 8: Information and data quality: Concepts and measuring

Data quality — Part 61: Data quality management: Process reference model

Data quality — Part 110: Master data: Exchange of characteristic data: Syntax, semantic encoding, and conform…

Data quality - Data quality assessment. Profiling