Criação de um indicador de qualidade de dados para problemas de modelagem preditiva

Abstract

Data Quality is a relevant subject in Data Mining Tasks. Wrong Results can be obtained when a poor data quality is selected for the task. Governance Frameworks used curren- tly by the community of practice in Data Mining are very generic and abstract as they should be: though they emphasize the importance of the exploration of the dataset and the related issues with data quality, they don´t supply a complete framework for evalu- ating the data quality. This work proposes a numeric index for data quality evaluation, using Descriptive Statistics tools to quantitatively evaluate data quality in a methodic and structured way. The proposed index evaluates problems with data sample, giving them a weighted score and finally gives an average in order to indicate if further cleansing and adjustment are needed for the dataset before proceed with the Data Mining Tasks.