Um estudo comparativo de técnicas de detecção de outliers no contexto de classificação de dados

Abstract

Outliers are objects that deviate considerably from others in relation to some measure, and promote great influence in the analysis of the data. In statistics, this influence may induce an equivocal analysis of the data, in which case the outliers constitute data that need to be removed. For other applications, the outlier may represent some valuable information, dealing with some type of fraud, system intrusion, computer network anomalies, mechanical failures and critical clinical condition. In any case, outliers need to be identified, regardless of their treatment. The literature provides several techniques for detection of outliers, each with its characteristics and specificities, which in turn have been applied in several domains, in order to solve singular problems. To specify which technique performs better for a particular data domain is a challenge that is still little explored in the literature and causes the development of strategies to measure the performance of outliers detection techniques. In this sense, the proposal of this work is to present a comparative study of outliers detection techniques, through a methodology that allows a uniform and objective analysis. The techniques used in the comparative analysis are distributed in techniques based on statistical methods, proximity and distance. As part of the methodology, they are applied in the pre-processing of the data, where their performance is measured by analyzing the effect of this application on the classifier induction. Classifier evaluation metrics serve as performance indicators for classifiers. According to the results of the experiments, it was possible to effectively analyze the performance of outliers detection techniques for different domains, and confirm the validity of the methodology