Detecting topics in documents by clustering word vectors
Tipo
Artigo de evento
Data de publicação
2020
Periódico
Advances in Intelligent Systems and Computing
Citações (Scopus)
9
Autores
de Miranda G.R.
Pasti R.
de Castro L.N.
Pasti R.
de Castro L.N.
Orientador
Título da Revista
ISSN da Revista
Título de Volume
Membros da banca
Programa
Resumo
© Springer Nature Switzerland AG 2020.The automatic detection of topics in a set of documents is one of the most challenging and useful tasks in Natural Language Processing. Word2Vec has proven to be an effective tool for the distributed representation of words (word embeddings) usually applied to find their linguistic context. This paper proposes the use of a Self-Organizing Map (SOM) to cluster the word vectors generated by Word2Vec so as to find topics in the texts. After running SOM, a k-means algorithm is applied to separate the SOM output grid neurons into k clusters, such that the words mapped into each centroid represent the topics of that cluster. Our approach was tested on a benchmark text dataset with 19,997 texts and 20 groups. The results showed that the method is capable of finding the expected groups, sometimes merging some of them that deal with similar topics.
Descrição
Palavras-chave
Assuntos Scopus
Automatic Detection , Distributed representation , Effective tool , K-means , NAtural language processing , Topic detection , Word vectors , Word2Vec