Repository | Book | Chapter
(2017) Research and advanced technology for digital libraries, Dordrecht, Springer.
A comparison of pre-processing techniques for twitter sentiment analysis
Dimitrios Effrosynidis, Symeon Symeonidis, Avi Arampatzis
pp. 394-406
Pre-processing is considered to be the first step in text classification, and choosing the right pre-processing techniques can improve classification effectiveness. We experimentally compare 15 commonly used pre-processing techniques on two Twitter datasets. We employ three different machine learning algorithms, namely, Linear SVC, Bernoulli Naïve Bayes, and Logistic Regression, and report the classification accuracy and the resulting number of features for each pre-processing technique. Finally, based on our results, we categorize these techniques based on their performance. We find that techniques like stemming, removing numbers, and replacing elongated words improve accuracy, while others like removing punctuation do not.
Publication details
DOI: 10.1007/978-3-319-67008-9_31
Full citation:
Effrosynidis, D. , Symeonidis, S. , Arampatzis, A. (2017)., A comparison of pre-processing techniques for twitter sentiment analysis, in J. Kamps, G. Tsakonas, Y. Manolopoulos, L. Iliadis & I. Karydis (eds.), Research and advanced technology for digital libraries, Dordrecht, Springer, pp. 394-406.
This document is unfortunately not available for download at the moment.